Archive

Archive for the ‘Quartex Pascal’ Category

When Nicola Tesla slaps you over the head

March 24, 2021 Leave a comment

If you have poked around the Quartex Pascal RTL you might have notice that QTX comes with a serial-number minting system. Creating unique serial numbers that can be mathematically re-traced back to a root key (number sequence) is extremely tricky business. My solution was to dip my fingers into esoteric and occult numerology, because our ancient forbearers had a knack for puzzles and number based riddles.

And here I thought I was super clever, only to discover that Nicola Tesla scribbled a similar system on a napkin back in the late 1800s (figuratively speaking). Turns out that the basis of my system is more or less identical to Tesla’s numbers and ultimately bound by their relationships, where you operate with a growth factor that is a multiple of 12, modulated and held in check by Fibonacci, Lucas or Leonardo sequencing.

So my ego got a well deserved slap (which is always healthy, we should all be humble in the presence of that man).

I have never really been that interested in Tesla until recently, and the little I have read about him makes me incredibly sad. This man was not decades ahead of his time, but centuries.

In my view, the biggest tragedy in human history is without a doubt the loss of the great library in Alexandria, Egypt. Second only with the murder of Hypatia; a series of events that would eventually catapult humanity as a whole into a dark-age that lasted for 2000 years.

But having spent some time this morning reading about Tesla, I would add him to that list of tragic events that have affected our history (or in his case, being prevented from lifting mankind up). This is a man that constructed the walkie-talkie in the late 1800s. He even theorized that both audio and video could be transmitted over a hand-held device. And this was in the late 1800s (!).

Tesla’s analysis of numbers, based on multiples of 12, each segment seeding the next
From the Quartex Pascal IDE, the serial number minting dialog

Above: The serial-number minting dialog from the IDE. Here we use 12 seed numbers to form the root key, and each serial number is grown from these using natural numbers, as employed by various mystics and esoteric traditions.

Hat off Tesla. It is a great shame that you were born into a world that neither understood or appreciated the wonders you delivered.

Nicolas Tesla’s notebooks is best read on your knees.

Now I need to scotch tape my ego back together and get to work.

HTMLComponents for Delphi, a closer look

November 25, 2020 4 comments

For a while now I have been paying attention to Alexander Sviridenkov’s components for Delphi. First of all because Delphi doesn’t really have that many options when dealing with HTML beyond embedding the classical browsers (Chromium, Firefox or Edge); And while Dave Baldwin’s work has served the community well in the past, TFrameBrowser is 20 years old. So there is definitively room for improvement.

Secondly, in my work with Quartex Pascal, a system that compiles Object Pascal to JavaScript for HTML5, being able to work with HTML and render HTML easily from my IDE is obviously interesting. The form designer in particular could benefit from showing “live” graphics rather than representative rectangles.

Quartex Pascal uses HTMLComponents in a variety of places to render UI elements

All of that is possible to do with Chromium if you run it in an off-screen capacity, but getting good results is very tricky. Chromium Embedded runs in a separate thread (actually, multiple threads) and sharing video memory, injecting HTML to avoid a full reload — lets just say that a Delphi native component package would make all the difference. Enter HTMLComponents.

Focus on the essentials first

The way that Alexander has proceeded with his components can resemble my own philosophy (or indeed anyone who has been a developer for a while). It’s the kind of work approach you end up with through experience, namely, to start with the basics and make sure that is rock solid (read: focus on the foundation code, that’s what’s going to matter the most. Trust me). It’s so tempting to run off on a tangent, adding more and more functionality – typically visually pleasing stuff, but as mature developers will know, if you go down that path what you end up with is a very expensive mess.

HTMLComponents have some high profile customers. Here used in Help & Manual

Thankfully, Alexander has gone about his work in an orderly, no-nonsense way. He began with the HTML parser, making sure that was flexible, compliant and delivered great performance (over 100 Mb a second!). Then he moved on to the CSS part of the equation and implemented a high performance styling engine. The reason I outline this is because I don’t think people fully grasp the amount of work involved. We think of HTML as a simple tag based format, but the sheer infrastructure you need to represent modern HTML properly is enormous. There is a reason Delphi component vendors shy away from this task. Thankfully Alexander is not one of them.

Scripting?

Next we have the scripting aspect. And here is the twist, if we can call it that. HTMLComponents is not written to be a browser. It is written to enable you to render HTML5 at high speed within a native application, including CSS animations and Tweening (a technique made popular by Apple. Like sliding forms or bouncing swipe behavior).

In other words, if you are expecting to find a browser, something like Dave Baldwin’s now ancient TFrameBrowser, then you should probably look to the new TEdgeBrowser component from Embarcadero. So JavaScript is not yet natively supported. HTMLComponents falls into the category of a UI presentation framework more than a browser.

If however, like myself, you want to handle presenting HTML5, PDF, RTF and Word documents without a ton of dependencies (Chromium adds 150Mb worth of libraries you need to ship), provide your users with a compliant HTML WYSIWYG Editor – and also deliver those fancy animated UI elements – then you are going to love HTMLComponents.

I should mention that HTMLComponents has its own scripting engine, but it’s not JavaScript. But for those situations where a script is needed, you can tap into the scripting engine if you like. Or deal with everything natively. It’s your choice.

Document editor

Pretty impressive list of controls

The reason I mentioned Alexander’s architecture and how his codebase has evolved, is because a high performance document rendering engine can be very useful for a variety of tasks.

One thing is rendering HTML5 with all the bells and whistles that entails, but what about RTF? What about Word documents? What about PDF documents? Once you have a rock solid engine capable of representing HTML5, the next logical step is to branch out and work with the other formats of our times. And that is just what Alexander did.

But before we look at those features, let’s have a peek at what components you get.

As you can see from the picture above, HTMLComponents is not just about drawing HTML. Being able to represent HTML is useful in a variety of situations since it simplifies visual feedback that would otherwise be very time consuming to implement. So instead of limiting that power to a single control, HTMLComponents come with common controls that have been infused with superior graphical powers.

Full editor, and TAction’s for all features!

The most powerful component in the above list is without a doubt the HTML editor component (also notice that the package installs both standard and DB variations of the various controls). This is quite simply a fully compliant WYSIWYG editor – complete with all the formatting features you expect.

  • WYSIWYG editing.
  • Does not use IE or other libraries (100% native Delphi code).
  • Supports all Delphi versions from Delphi 5 to Delphi 10.4 Sydney.
  • Supports Lazarus (Windows/Linux)
  • VCL (Win32/64) FMX (Windows / OSX / Android / iOS / Linux)
  • Full support for touch-screen devices – gestures, text selection (Windows Tablets/Screens, iOS, Android, OSX)
  • Smooth scrolling on Android and iOS.
  • Unicode support for Delphi 6 – 2007 (requires TNTUnicode).
  • Scalable (High DPI  support).
  • Live spellchecking and autocorrection (built-in support for Addict).
  • Live text styles preview (font family,size, color, background).
  • RTF and MS Word DOCX Import on all platforms.
  • PDF export on Windows, Android, OSX and iOS.
  • DB-Aware version
  • Full support for HTML tags and CSS properties.
  • Full access from Delphi code to DOM and Styles.
  • Images, lists, blocks, font styles
  • Tables support
  • Print and Print Preview
  • Embedded Find dialog, Text search, Document Index generation.
  • Copy from/paste to MS Word, browsers and other applications
  • Embedded Markdown, Pascal and HTML syntax highlighting.
  • HTML-based editor controls (HtFontCombo, HtFontSizeCombo, HtColorCombo, HtTableAddCombo, HtBorderCombo, HtTableBorderCombo)

That is a solid list of features, and did I mention you get full source-code?

HTML empowered controls

If you are looking over the list of controls above and expecting to find something like a browser or viewer control, you won’t find it. The closest thing to a HTML viewer is the panel control (THtPanel). It exposes properties and methods to populate it with HTML (as does all the controls), set what type of scrollbars you need (if any), how to deal with links, images and CSS styling – and then it’s a matter of feeding some HTML into the control.

Quartex Pascal is still under heavy development. Here the project build options is handled via HTMLComponents. This means styling and “bling” can be easily added once we reach a release candidate. I always keep it simple for as long as I can.

Obviously controls like THtCombobox have behavior that is dictated by the OS, but you can style the child elements (rows for example) being displayed, the border etc. using the full might of HTML5. And yes, you can apply CSS transitions there as well – which is (excuse my french) bloody spectacular!

I mentioned that HTMLComponents were not designed to be a browser replacement, but rather to make it easier for native developers to tap into the design power and visual feedback that makes HTML5 so productive to use for UIs. Well, once you have set the properties for a panel and given it some HTML -you can do some pretty amazing things!

CSS transforms galore! Hover-effects, fading, overlapping, gradients, transitions – the works!

HTML takes a lot of grunt work out of the equation for you. For example, let’s say you wanted to produce a demo like the one in the picture above (see direct link in the next paragraph). With all the effects, transitions, pictures and displacement elements. Just how much work would that be in traditional Delphi or C++ ?

Note: You can download the Demo application directly, here:
https://delphihtmlcomponents.com/csstransforms.zip

First you would need a panel container for each picture, then a canvas to hold the picture, then you would need to handle the interaction events- and finally access the canvas to draw the many alpha blended transitions (the picture here really doesn’t do the framework credit, you have to see them to fully appreciate the level of detail and performance HTMLComponents delivers). And all of that is before you have new elements flying in from the sides or above, that fades perfectly with the backdrop. All of it working according to a timeline (tweening as its called).

Instead of all that work, having to write a tweening engine, 32 bit alpha-blending DIBs (device independent bitmaps), deal with god knows how much work — you can just deliver some HTML and CSS and let HTMLComponents handle all of it. With zero external dependencies I might add! This is a pure Delphi library. There are no references to external browser controls or anything of the kind. HTMLComponents does exactly what it says on the box – namely to render HTML5 at high speed. And it delivers.

Here is the HTML for one of the pictures with effects in the demo:

<div class="view view-sixth">
  <img src="images/13.jpg" />
  <div class="mask">
    <h2>Hover Style #6</h2>
    <p>A wonderful serenity has taken possession ..</p>
    <a href="#" class="info">Read More</a>
  </div>
</div>                                            

And here is the CSS animation transition code for the same. Please note that the original code contained definitions for IE, Opera, Webkit and Firefox. I removed those for readability:

.view-sixth img {
   transition: all 0.4s ease-in-out 0.5s;
}
.view-sixth .mask {
   background-color: rgba(146,96,91,0.5);
   filter: alpha(opacity=0);
   opacity: 0;
   transition: all 0.3s ease-in 0.4s;
}
.view-sixth h2 {
   filter: alpha(opacity=0);
   opacity: 0;
   border-bottom: 1px solid rgba(0, 0, 0, 0.3);
   background: transparent;
   margin: 20px 40px 0px 40px;
   transform: scale(10);
   transition: all 0.3s ease-in-out 0.1s;
}
.view-sixth p {
   filter: alpha(opacity=0);
   opacity: 0;
   transform: scale(10);
   transition: all 0.5s ease-in-out 0.2s;
}
.view-sixth a.info {
   filter: alpha(opacity=0);
   opacity: 0;
   transform: translateY(100px);
   transition: all 0.3s ease-in-out 0.1s;
}
.view-sixth:hover .mask {
   filter: alpha(opacity=100);
   opacity: 1;
   transition-delay: 0s;
}
.view-sixth:hover img {
   transition-delay: 0s;
}
.view-sixth:hover h2 {
   filter: alpha(opacity=100);
   opacity: 1;
   transform: scale(1);
   transition-delay: 0.1s;
}
.view-sixth:hover p {
   filter: alpha(opacity=100);
   opacity: 1;
   transform: scale(1);
   transition-delay: 0.2s;
}
.view-sixth:hover a.info {
   filter: alpha(opacity=100);
   opacity: 1;
   transform: translateY(0px);
   transition-delay: 0.3s;
}		

If CSS is not something you normally don’t bother with, the code above might look complex and alien. But there are tons of websites that have wizards, tutorials and even online editors (!), so if you take the time to read up on how CSS transitions work (they are quite easy), you will knock out some impressive effects in no time.

Once you have built up a collection of such effects, just link it into your Delphi application as a resource if you don’t want external files. Personally I think its a good thing to setup the UI in separate files like that, because then you can update the UI without forcing a binary installation on your customers.

So if we consider the amount of Delphi code we would have to write to deliver the same demo using stock VCL, sum up the cost in hours – and most likely the end result as well (Alexander is exceptionally good at graphical coding), I for one cant imagine why anyone would ignore HTMLComponents. I mean serious, you are not going to beat Alexander’s code here. And why would you waste all that time when you can buy ready to use controls with source-code for such a modest price?

Office formats

I mentioned briefly that with a powerful document rendering engine in place, that the next step of the way would be to cover more formats than just HTML. And this is indeed what Alexander has done.

If you invest in his Add-On Office package for HTMLComponents, you will be able to load and display a variety of document formats. And just like HTMLComponents the code is 100% Delphi and has zero dependencies. There are no COM objects or ActiveX bindings involved. Alexander’s code loads, parses and converts these documents instantly to HTML5, and you can view the results using HTMLComponents or in any modern browser.

Following document formats are supported:

  • Rich Text Format (RTF)
  • MS Word 6-2007 binary format (DOC)
  • MS Word XML document (DOCX)
  • MS Power Point binary format (PPT)
  • MS Power Point XML format (PPTX)
  • MS Excel binary format (XLS)
  • MS Excel XML format (XLSX)
  • Adobe PDF format (PDF)
  • Supercalc format (SXC)
  • EPUB (electronic books).

Besides the document conversion classes you also get the following code, which is pretty hard-core and useful:

  • EMF/WMF to SVG conversion
  • TTF to WOFF conversion
  • TTF normalization
  • TTF to SVG conversion
  • CFF to TTF conversion
  • Adobe PostScript to TTF conversion.

For me this was a god-send because I was using Gnostice’s PDF viewer to display the documentation for Quartex Pascal in the IDE. Being able to drop that dependency (and cost!) and use HTMLComponents uniformly throughout the IDE makes for a much smaller codebase – and cleaner code.

Final thoughts

The amount of code you get with HTMLComponents is quite frankly overwhelming. One thing is dealing with a tag based format, but once you throw special effects, transitions and standards into the mix – it quickly becomes a daunting task. But Alexander is delivering one of the best written component packages I have had the pleasure of owning. If you need a fresh UI for your application, be it POS, embedded or desktop utilities – HTMLComponents will significantly reduce the time spent.

I should also underline that HTMLComponents also works on FMX and Mobile devices ( Windows, OS X, Android, iOS and Linux even!). I’m not a huge fan of FMX myself so being able to design my forms using HTML and write event handlers in native Delphi is perfect. FMX has a lot of power, but the level of detail involved can be frustrating. HTMLComponents takes the grunt out of it, so I can focus on application specific tasks rather than doing battle with the UI.

The only thing I would like to see added, is support for JavaScript. HTMLComponents makes it easy for you to intercept scripts and deal with them yourself (HTMLComponents also have a pascal inspired script), but I do hope Alexander takes the time to add Besen (a native Delphi JavaScript engine) as an option. It really is the only thing I can think of in the “should have” apartment. Everything else is already in there.

I have to give HTMLComponents 9 out of 10 stars. It would have scored a perfect 10 with JS support. But this is the highest score I have ever given on my blog, so that’s astronomical. Well done Alexander! I look forward to digging into the office suite in the weeks ahead, and will no doubt revisit this topic in further articles.

Visit Alexander’s website here: https://www.delphihtmlcomponents.com/index.html

Quartex Pascal Build 13b ships

September 30, 2020 4 comments

While it can come across as disingenuous, I frickin love this project! As a developer yourself you know that feeling, when you manage to unify various aspects of your program, so that it all fits just perfectly. And the way I implemented file-handling and editors is within that sweet spot.

What is new?

It’s been a couple of weeks since I posted here last, so the list of changes will be quite dramatic. I think its best to focus on the highlights or this post would become very long!

Ironwood license management

Up until 2018 one of my products was a component package called HexLicense. This is a component package for Delphi that provides serial number validation, license handling and (most importantly) serial number minting. The HexLicense components were sold commercially until 2018 when I took them off the market and open-sourced (access was via Patreon. It is now moved to the Quartex Pascal project instead).

Ironwood is now an intrinsic part of the RTL and IDE

Im not going to go into how difficult it is to produce thousands of distinctly different serial numbers based on seed data, but it’s no walk in the park.

The final implementation I made for license minting and validation, was called Ironwood. It took the engine behind HexLicense and took it to a completely new level, incorporating both obfuscation and number modulation.

Generating license-number batches is literally one mouse-click to achieve

Needless to say, Ironwood is now a part of the Quartex Pascal RTL. To make it easier to work with the IDE has a nice utility for generating license-numbers, loading and saving keys, exporting license number batches – and much more.

There is also a ready-to-rock node.js application that can generate keys from the command-line (which is good to invoke from a server or service, so that it executes as a separate process).

HTML structure provider

The IDE has a very clean internal architecture, where the actual work is isolated in a set of easy to understand classes. One of these classes is called a TIDEAstProvider class. This is a class whose job it is to parse and otherwise work with whatever content an editor has, and deliver symbolic information that can be displayed in the file-structure treeview.

The IDE provides structured parsing and also Tag suggestions. More clever functionality will be added as we move into the final phases.

Obviously we have an object pascal provider, which will quickly compile and generate an AST very quickly in memory. This is used to power both the structure treeview and the code-suggestion.

Next, we have the exact same provider for JavaScript. So when you open a JavaScript file, the file will be processed to produce an AST, and the symbol information will be displayed exactly like your object pascal is. So behavior between these are identical.

We now also have a HTML provider, with a CSS provider on the way. The HTML provider is still in its infancy, but its flexible enough to represent a good foundation to work with. So I will no doubt return to this task later to smarten the provider logic up, and better handle un-valid HTML and CSS.

Code suggestion

Code suggestion is a pretty standard function these days. We have had support for this under Object Pascal for a while now in the IDE (with JavaScript on the way).

Note: the code suggestion-box is un-styled at this point. Custom painting will be added once the core functionality is done.

The IDE displays both a structural view of the unit, as well as in-depth code suggestion

Code suggestion for HTML is now in place too. It needs a bit of polish since the rules for HTML are wildly different from a programming language, but common behavior like TAG suggestion is there — with attributes, properties and events to follow.

So even if you are not an object pascal developer, the IDE should be nice to work with for traditional JavaScript / HTML code.

Form Recognition

While we cannot activate the form-designer just yet, since we need more AST functionality to extract things like class properties and attributes “live” to be able to do that properly — we are getting really close to that milestone.

The IDE however now recognize form files, so if your unit has an accompanying DFM file, the IDE is smart enough to open up a form-page. Form pages are different from ordinary pascal pages, since they also have the form designer control on a sub-tab. More or less identical to Delphi and Lazarus.

When you open a form-unit, the IDE is smart enough to recognize it as a form, opening the file-pair up in a layout capable page, just like Delphi

It is going to be so nice to get the form-designer activated. Especially the stack-based layout, which makes scalable, dynamic layout easy to create and work with.

The QTX RTL also supports orientation awareness as a part of the visual component system. One of the first things you will notice when exploring the code, is that ReSize() ships in an Orientation parameter, so you can adjust your layout accordingly.

Help and documentation inside the IDE

The IDE now has a PDF viewer with search functionality built-in. So when you click on Help and Documentation, a tab which shows the documentation PDF opens. This makes it easy to read, learn and find the information you need fast.

The IDE now has it’s own PDF renderer, so you can read the documentation directly

Well, that was a brief overview of what has changed since last time!

Next update is, as always, the weekends. We tend to land on sundays for new binaries, but do issue hotfixes in the evenings (weekdays) if something critical shows up.

Come join the fun!

Want to support the project? All financial backers that donates $100+ get their name in the product, access to the full IDE source-code on completion, and access to the Quartex Media Desktop system (which is a complete web desktop with a clustered back-end,  compiled to JavaScript and running on node.js. Portable, platform and chipset independent, and very powerful).

A smaller sum monthly is also welcome. The project would not exist without members backing it with $100, $200 etc every month. This both motivates and helps me allocate hours for continuous work.

When the IDE is finished you will also have access to the IDE source-code through special dispensation. Backers have rights since they have helped create the project.

Your help matters! It pays for components, hours and above all, tools and motivation. In return, you get full access to everything and a perpetual license. No backers will ever pay a cent for any future version of Quartex Pascal. Note: PM me after donating so I can get you added to the admin group! Click here to visit paypal: https://www.paypal.com/paypalme/quartexNOR

All donations are welcome, both large and small. But donations over $100, especially reoccurring, is what drives this project forward.

Remember to send me a message on Facebook so I can add you to the Admin group: https://www.facebook.com/quartexnor/

Quartex Pascal, Build 10b is out for backers

July 26, 2020 5 comments

I am deeply moved by some of the messages I have received about Quartex Pascal. Typically from people who either bought Smart Mobile studio or have followed my blog over the years. In short, developers that like to explore new ideas; people who also recognize some of the challenges large and complex run-time libraries like the VCL, FMX and LCL face in 2020.

Since I started with all this “compile to JavaScript” stuff, the world has changed. And while I’m not always right – I was right about this. JavaScript will and have evolved into a power-house. Largely thanks to Microsoft killing Basic. But writing large scale applications in JavaScript, that is extremely time consuming — which is where Quartex Pascal comes in.

support2

Quartex Pascal evolves every weekend. There are at least 2 builds each weekend for backers. Why not become a backer and see the product come to life? Get instant access to new builds, the docs, and learn why QTX code runs so much faster than the alternatives?

A very important distinction

Let me first start by iterating what I mentioned in my previous post, namely that I am no longer involved with The Smart Company. Nor am I involved with Smart Mobile Studio. I realize that it can be difficult for some to separate me from that product, since I blogged and created momentum for it for more than a decade. But alas, life change and sometimes you just have to let go.

The QTX Framework which today has become a fully operational RTL was written by me back between 2013-2014 (when I was not working for the company due to my spinal injury). And the first version of that framework was released under an open-source license.

When I returned to The Smart Company, it was decided that to save time – we would pull the QTX Framework into the Smart RTL. Since I owned the QTX Framework, and it was open source, it was perfectly fine to include the code. The code was bound by the open source licensing model, so we broke no rules including it. And I gave dispensation that it could be included (although the original license explicitly stated that the units should remain untouched and separate, and only inherited from).

desktop_02

Quartex Media Desktop, a complete desktop system (akin to X Windows for Linux) written completely in Object Pascal, including a clustered, service oriented back-end. All of it done in Quartex Pascal  — a huge project in its own right. Under Quartex Pascal,  this is now a project type, which means you can have your own cloud solution at the click of a button.

As I left the company for good before joining Embarcadero, The Smart Company and myself came to an agreement that the parts of QTX that still exists in the Smart Mobile Studio RTL, could remain. It would be petty and small to make a huge number out of it, and I left on my own terms. No point ruining all that hard work we did. So we signed an agreement that underlined that there would be overlaps here and there in our respective codebases, and that the QTX Framework and Quartex Media Desktop was my property.

Minor overlaps

As mentioned there will be a few minor overlaps, but nothing substantial. The class hierarchy and architecture of the QTX RTL is so different, that 90% of the code in the Smart RTL simply won’t work. And I made it that way on purpose so there would be no debates over who made what.

QTX represents how I felt the RTL should have been done. And its very different  from where Smart Mobile Studio ended up.

The overlaps are simple and few, but it can be helpful for Smart developers to know about if they plan on taking QTX for a test-drive:

  • TInteger, TString and TVariant. These were actually ported from Delphi (part of the Sith Library, a pun on Delphi’s Jedi Library).
  • TDataTypeConverter came in through the QTX Framework. It has been completely re-written from scratch. The QTX version is endian aware (works on both ARM, X86 and PPC). Classes that deal with binary data (like TStream, TBuffer etc) inherit from TDataTypeConverter. That way, you dont have to call a secondary instance just to perform conversion. This is easier and much more efficient.
  • Low-level codecs likewise came from the QTX Framework, but I had to completely re-write the architecture. The old model could only handle binary data, while the new codec classes also covers text based formats. Codecs can be daisy-chained so you can do encoding, compression and encryption by feeding data into the first, and catching the processed data from the last codec in the chain. Very handy, especially when dealing with binary messages and database drivers.
  • The in-memory dataset likewise came from the QTX Framework. This is probably the only unit that has remained largely unchanged. So that is a clear overlap between the Smart RTL and QTX.
  • TextCraft is an open source library that covers Delphi, Freepascal and DWScript. The latter was pulled in and used as the primary text-parser in Smart. This is also the default parser for QTX, and have been largely re-written so it could be re-published under the Shareware license.

Since the QTX RTL is very different from Smart, I haven’t really bothered to use all of the old code. Stuff like the CSS Effects units likewise came from the QTX Framework, but the architecture I made for Smart is not compatible with QTX, so it makes no sense using that code. I ported my Delphi tweening library to DWScript in 2019, which was a part of my Patreon project. So most of the effects in QTX use our own tweening library. This has some very powerful perks, like being able to animate a property on any object rather than just a HTML Element. And you can use it for Canvas drawing too, which is nice.

Progress. Where are we now?

So, where am I in this work right now? The RTL took more or less 1 year to write from  scratch. I only have the weekends  for this type of work,  and it would have been impossible without my backers. So I cannot thank each backer enough for the faith in this. The RTL and new IDE is actually just a stopping point on the road to a much bigger project, namely CloudForge, which is the full IDE running as an application on the Quartex Media Desktop. But first, let’s see what has been implemented!

AST unit view

unit_view

The Unit Overview panel. Easy access to ancestor classes as links (still early R&D). And the entire RTL on a second tab. This makes it very easy to learn the new RTL. There is also proper documentation, both as PDF and standard helpfile.

When the object-pascal code is compiled by DWScript, it goes through a vigorous process of syntax checking, parsing, tokenizing and symbolization (or objectification is perhaps a better word), where every inch of the code is transformed into objects that the compiler can work with and produce code from. These symbols are isolated in what is known as an AST, short for “Abstract Symbol Tree”. Basically a massive in-memory tree structure that contains your entire program reduced to symbols and expressions.

In order for us to have a live structural view of the current unit, I have created a simple background process that compiles the current unit, grabs the resulting AST, locates the unit symbol, and then displays the information in a meaningful way. This is the exact same  as most other IDE’s do, be it Visual Studio, Embarcadero Delphi, or Lazarus.

So we have that in place already. I also want to make it more elaborate, where  you can click yourself to glory by examining ancestors, interfaces, partial class groups – as well as an option to include inherited members (which should be visually different). Either way, the AST code is done. Just need to consolidate a few tidbits so that each Treeview node retains information about source-code location (so that when you double-click a symbol, the IDE navigates to where the symbol exists in the codebase).

JavaScript parsing and compilation

QTX doesn’t include just one compiler, but three. In order for the unit structure to also work for JavaScript files I have modified Besen, which is an ES5 compatible JavaScript engine written in Delphi. And I use this much like DWScript to parse and work with the AST.

unit_view2

Besen is a wonderful compiler. Where DWScript produces JavaScript from Object Pascal, Besen produces bytecodes from JavaScript (which are further JIT compiled). This opens up for some interesting options. I need to add support for ES6 though, modules and require are especially important for modern node.js programming (and yes, the QTX RTL supports these concepts)

HTML5 Rendering and CSS preview

Instead of using Chromium inside the IDE, which is pretty demanding, I have decided to go for HTMLComponents to deal with “normal” tasks. The “Welcome” tab-page for example — it would be complete overkill to use a full Chromium instance just for that, and TEdgeBrowser is likewise shooting sparrows with a Bazooka.

THTMLComponents have a blistering fast panel control that can render more or less any HTML5 document you throw at it (much better than the old TFrameViewer component). But obviously, it doesn’t have JS support. But we won’t be using JS when displaying static information – or indeed, editing HTML5 compliant content.

WYSIWYG Editor

The biggest benefit for HTMLComponents, is that it’s a fully operational HTML compliant editor. Which means you can do more or less all your manual design with that editor. In Quartex Pascal there is direct support for HTML files. Quartex works much like Visual Studio code, except it has visual designers. So you can create a HTML file and either type in the code manually, or switch to the HTMLComponents editor.

Which is what products like Help & Manual uses it for

helpmanual3

Image from HTMLComponents application gallery website

Support for HTML, CSS and JS files directly

While not new, this is  pretty awesome. Especially since we can do a bit of AST navigation here too to present similar information as we do for Object Pascal. The whole concept behind the QTX RTL, is that you have full control over everything. You can stick to a normal Delphi like form designer and absolute positioning, or you can opt for a more dynamic approach where you create content via code. This is perfect for modern websites that blend scrolling, effects and content (both dynamic and static content) for a better user experience.

You can even (spoiler alert), take a piece of HTML and convert it into visual controls at runtime. That is a very powerful function, because when doing large-scale, elaborate custom controls – you can just tell the RTL “hey, turn this piece of HTML into a visual control for me, and deliver it back when you are ready).

Proper Form Designer

Writing a proper form designer like Delphi has is no walk in the park. It has to deal not just with a selected control, but also child elements. It also has to be able to select multiple elements based on key-presses (shift + click adds another item to the selection),  or from the selection rectangle.

unit_view3

A property form layout control. Real-time rendering of controls is also possible, courtesy of HTMLComponents. But to be honest, it just gets in the way. Its much easier to work with this type of designer. It’s fast, responsive, accurate and will have animated features that makes it a joy to work with. 

Well, that’s not going to be a problem. I spent a considerable amount of time writing a proper form designer, one that takes both fixed and dynamic content into account. So the Quartex form designer handles both absolute and stacked layout modes (stacked means top-down, what in HTML is knock as blocking element  display, where each section stretch to the full width, and only have a defined height [that you can change]).

Node.js Service Protocol Designer

Writing large-scale servers, especially clustered ones, is very fiddly in vanilla JavaScript under node.js. It takes 3 seconds to create a server object, but as we all know, without proper error handling, a concurrent message format, modern security and a solid framework to handle it all — that “3 second” thing falls to the ground quickly.

This is where the Ragnarok message system comes in. Which is both a message framework, and a set of custom servers adapted for dealing with that type of data. It presently supports WebSocket, TCP and UDP. But expanding that to include REST is very easy.

support3

This is where the full might of the QTX Framework begins to dawn. As i wrote before we started on the Quartex Media Desktop (Which this IDE and RTL is a part of), in the future developers wont just drag & drop components on a form; they will drag & drop entire ecosystems ..

But the power of the system is not just in how it works, and how you can create your own protocols, and then have separate back-end services deal with one part of your infrastructure’s workload. It is because you can visually design the protocols using the Node Builder. This is being moved into the QTX IDE as I type. So should make it for Build 12 next weekend.

In short, you design your protocols, messages and types – a bit like RemObjects SDK if you have used that. And the IDE generates both server and client code for you. All you have to do is fill in the content that acts on the messages. Everything else is handled by the server components.

Suddenly, you can spend a week writing a large-scale, platform agnostic service stack — that would have taken JavaScript developers months to complete. And instead of having to manage a 200.000 lines codebase in JavaScript — you can enjoy a 4000 line, easily maintainable Object Pascal codebase.

Build 11

Im hoping to have build 11 out tomorrow (Sunday) for my backers. Im still experimenting a bit with the symbol information panel, since I want it to be informative not just for classes, but also for methods and properties. Making it easy to access ancestor implementations etc.

I also need to work a bit more on the JS parsing. Under ES5 its typical to use variables to hold objects  (which is close to how we think of a class), so composite and complex datatypes must be expanded. I  also need to get symbol position to work property, because since Besen is a proper bytecode compiler, it doesn’t keep as much information in it’s AST as DWScript does.

Widgets (which is what visual controls are called under QTX) should appear in build 12 or 13. The IDE supports zip-packages. The file-source system I made for the TVector library (published via Embarcadero’s website a few months back) allows us to mount not just folders as a file-source, but also zip files. So QTX component packages will be ordinary zip-files containing the .pas files, asset files and a metadata descriptor file that tells the IDE what to expect. Simple,  easy and very effective.

Support the project!

Want to support the project? All financial backers that donates $100+ get their name in the product, access to the full IDE source-code on completion, and access to the Quartex Media Desktop system (which is a complete web desktop with a clustered back-end,  compiled to JavaScript and running on node.js. Portable, platform and chipset independent, and very powerful).

support

Your help matters! It pays for components, hours and above all, tools and motivation. In return, you get full access to everything and a perpetual license. No backers will ever pay a cent for any future version of Quartex Pascal. Note: PM me after donating so I can get you added to the admin group! Click here to visit paypal: https://www.paypal.com/paypalme/quartexNOR

All donations are welcome, both large and small. But donations over $100, especially reoccurring, is what drives this project forward. It also gets you access to the Quartex Developer group on Facebook where new builds, features etc is happening. It is the best way to immediately get the latest build, read documentation as its written and see the product come to life!

 

Quartex Pascal, convergence is near

July 16, 2020 1 comment

67582488_10156396548830906_5204248427029856256_o

A Quartex Cluster of 5 x ODroid XU4. A $400 super computer running Quartex media Desktop. Enough to power a school.

I only have the weekends to work on Quartex Pascal, but I have spent the past 18 months tinkering away, making up for wasted time. So I’m just going to leave some pictures here for you to enjoy.

Note: I was asked on LinkedIn if this has anything to do with Smart Mobile Studio, and the answer is a resounding no. I have nothing to do with Smart any more. QTX Pascal is a completely separate project that is written from scratch by yours truly.

The QTX Framework was initially a library I created back in 2014, but it has later been completely overhauled and turned into a full RTL. It is not compatible with Smart Pascal and has a completely different architecture.

QTX Pascal is indirectly funded by the Amiga Retro Community (which might sound strange, but the technical level of that community is beyond anything I have encountered elsewhere) since QTX is central to the creation of the Quartex Media Desktop. It is a shame that Embarcadero decided to not back the project. The compiler and toolchain would have been a part of Delphi by now, and I wouldn’t have to write a separate IDE. But when they see what this system can deliver in terms of services, database work, mobile and embedded -they might regret it. The project only accepts donation funding, I am not interested in investors or partners. If you want a vision turned into reality, you gotta do it yourself. Everything else just gets in the way.

For developers by developers

Quartex Pascal is made for the community. It will be free for students and open-source projects. And a commercial license will never exceed $300. It is a shareware license and the financial aspects is purely to help fund further research and development of the desktop cloud platform. The final goal (CloudForge) is to compile the IDE itself to JavaScript, so people only need a browser to write enterprise level applications via Quartex Media Desktop. When that is finished, my work is done – and people have a clear path to the future.

qtx_run_07

Unlike other systems, QTX started with the non-visual stuff, so the system has a well implemented infrastructure for writing universal services and servers, using node.js as a deployment host. Services are also Docker friendly. Runs without change on Windows, Mac OS, Linux and a wealth of embedded systems and SBCs (single board computers)

qtx_run_08

A completely new RTL written from scratch generates close to native speed JS, highly compatible (even with legacy browsers) and rock solid

qtx_run_09

There are several display modes for QTX forms, from dynamic to absolute positioning. You can mix and match between HTML and QTX code, including a HTML5 compliant WYSIWYG editor and style manager. Makes content handling a lot easier

qtx_run_10

Write object pascal, JavaScript, HTML, LDEF (webassembly), node.js services – or mix and match between them all for maximum potential. Writing mobile applications is now ridiculously easy compared to “other tools” out there.

Oh and for the proverbial frosting — The full clustered Quartex Media desktop and services is a project type. Thats right. A complete cloud infrastructure suitable for teams, kiosks, embedded, schools, intranets – and even an replacement OS for ChromeOS. You don’t need to interface with Amazon, you get your own Amazon (optional naturally).

desktop_02

Filesystem over websocket, IPC between hosted apps and desktop, full back-end services that are clustered, and run on anything from a Raspberry PI 4 to low-cost ARM SBCs.

49938355_1169526123220996_502291013608407040_o

Web Assembly made easy. Both for Delphi and QTX

smartdesk

Let there be rock

Oh, and documentation. Loads and loads of documentation.

qtx_run_11

Proper documentation, both class overview and explanations that a human being has written is paramount for learning and getting up to speed quickly.

I don’t have vacation this year, which means I only have weekends to tinker away. But i have spent the past 18-ish months preparing and slowly finishing the pieces I needed. From vector containers to form design controls, to a completely re-written RTL from scratch — so yeah. This time I’m doing it my way.

C/C++ porting, QTX and general status

March 15, 2020 3 comments

C is a language that I used to play around with a lot back in the Amiga days. I think the last time I used a C compiler to write a library must have been in 1992 or something like that? I held on to my Amiga 1200 for as long as i could – but having fallen completely in love with Pascal, I eventually switched to x86 and went down the Turbo Pascal road.

Lately however, C++ developers have been asking for their own Developer group on Facebook. I run several groups on Facebook in the so-called “developer” family. So you have Delphi Developer, FPC Developer, Node.JS Developer and now – C++Builder developer. The groups more or less tend to themselves, and the node.js and FPC groups are presently being seeded (meaning, that the member count is being grown for a period).

The C++Builder group however, is having the same activity level as the Delphi group almost, thanks to some really good developers that post links, tips and help solve questions. I was also fortunate enough to have David Millington come on the Admin team. David is leading the C++Builder project, so his insight  and knowledge of both language and product is exemplary. Just like Jim McKeeth, he is a wonderful resource for the community and chime in with answers to tricky questions whenever he has time to spare.

Getting back in the saddle

Having working some 30 years with Pascal and Object Pascal, 25 of those years in Delphi, C/C++ is never far away. I have an article on the subject that i’ve written for the Idera Community website, so I wont dig too deep into that here — but needless to say, Rad Studio consists of two languages: Object Pascal and C/C++, so no matter how much you love either language, the other is never far away.

So I figured it was time for this old dog to learn some new tricks! I have always said that it’s wise to learn a language immediately below and above your comfort zone. So if Delphi is your favorite language, then C/C++ is below you (meaning: more low level and complex). Above you are languages like JavaScript and C#. Learning JavaScript makes strategic sense (or use DWScript to compile Pascal to JavaScript like I do).

When I started out, the immediate language below Object Pascal was never C, but assembler. So for the longest time I turned to assembler whenever I needed a speed boost; graphics manipulation and processing pixels is especially a field where assembly makes all the difference.

But since C++Builder is indeed an integral part of Rad Studio, and Object Pascal and C/C++ so intimately connected (they have evolved side by side), why not enjoy both assembly and C right?

So I decided to jump back into the saddle and see what I could make of it.

C/C++ is not as hard as you think

intf

I’m having a ball writing C/C++, and just like Delphi – you can start where you are.

While I’m not going to rehash the article I have already prepared for the Idera Community pages here, I do want to encourage people to give it a proper try. I have always said that if you know an archetypal language, you can easily pick up other languages, because the archetypal languages will benefit you for a lifetime. This has to do with archetypal languages operating according to how computers really work; as opposed to optimistic languages (a term from the DB work, optimistic locking), also called contextual languages, like C#, Java, JavaScript etc. are based on how human beings would like things to be.

So I now had a chance to put my money where my mouth is.

When I left C back in the early 90s, I never bothered with OOP. I mean, I used C purely for shared libraries anyways, while the actual programs were done in Pascal or a hybrid language called Blitz Basic. The latter compiled to razor sharp machine code, and you could use inline assembly – which I used a lot back then (very few programmers on those machines went without assembler, it was almost given that you could use 68k in some capacity).

Without ruining the article about to be published, I had a great time with C++Builder. It took a few hours to get my bearings, but since both the VCL and FMX frameworks are there – you can approach C/C++ just like you would Object Pascal. So it’s a matter of getting an overview really.

Needless to say, I’ll be porting  a fair share of my libraries to C/C++ when I have time (those that makes sense under that paradigme). It’s always good to push yourself and there are plenty of subtle differences that I found useful.

Quartex Media Desktop

When I last wrote about QTX we were nearing the completion of the FileSystem and Task Management service. The prototype had all its file-handling directly in the core service  (or server) which worked just fine — but it was linked to the Smart Pascal RTL. It has taken time to write a new RTL + a full multi-user, platform independent service stack and desktop (phew!) but we are seeing progress!

desktop

The QTX Baseline backend services is now largely done

The filesystem service is now largely done! There are a few synchronous calls I want to get rid of, but thankfully my framework has both async and sync variations of all file procedures – so that is now finished.

To make that clearer: first I have to wrap and implement the functionality for the RTL. Once they are in the RTL, I can use those functions to build the service functions. So yeah, it’s been extremely elaborate — but thankfully it’s also become a rich, well organized codebase (both the RTL and the Quartex Media Desktop codebases) – so I think we are ready to get cracking on the core!

The core is still operating with the older API. So our next step is to remove that from the core and instead delegate calls to the filesystem to our new service. So the core will simply be reduced to a post-office or traffic officer if you like. Messages come in from the desktops, and the core delegates the messages to whatever service is in charge of them.

But, this also means that both the core and the desktop must use the new and fancy messages. And this is where I did something very clever.

While I was writing the service, I also write a client class to test (obviously). And the way the core works — means that the same client that the core use to talk to the services — can be used by the desktop as well.

So our work in the desktop to get file-access and drives running again, is to wrap the client in our TQTXDevice ancestor class. The desktop NEVER accesses the API directly. All it knows about are these device drivers (or object instances). Which is  how we solve things like DropBox and Google Drive support. The desktop wont have the faintest clue that its using Dropbox, or copying files between a local disk and Google Drive for example — because it only communicates with these device classes.

Recursive stuff

One thing that sucked about node.js function for deleting a folder, is that it’s recursive parameter doesn’t work on Windows or OS X. So I had to implement a full recursive deletefolder routine manually. Not a big thing, but slightly more painful than expected under asynchronous execution. Thankfully, Object Pascal allows for inline defined procedures, so I didn’t have to isolate it in a separate class.

Here is some of the code, a tiny spec compared to the full shabam, but it gives you an idea of what life is like under async conditions:

unit service.file.core;

interface

{.$DEFINE DEBUG}

const
  CNT_PREFS_DEFAULTPORT     = 1883;
  CNT_PREFS_FILENAME        = 'QTXTaskManager.preferences.ini';
  CNT_PREFS_DBNAME          = 'taskdata.db';

  CNT_ZCONFIG_SERVICE_NAME  = 'TaskManager';

uses
  qtx.sysutils,
  qtx.json,
  qtx.db,
  qtx.logfile,
  qtx.orm,
  qtx.time,

  qtx.node.os,
  qtx.node.sqlite3,
  qtx.node.zconfig,
  qtx.node.cluster,

  qtx.node.core,
  qtx.node.filesystem,
  qtx.node.filewalker,
  qtx.fileapi.core,

  qtx.network.service,
  qtx.network.udp,

  qtx.inifile,
  qtx.node.inifile,

  NodeJS.child_process,

  ragnarok.types,
  ragnarok.Server,
  ragnarok.messages.base,
  ragnarok.messages.factory,
  ragnarok.messages.network,

  service.base,
  service.dispatcher,
  service.file.messages;

type

  TQTXTaskServiceFactory = class(TMessageFactory)
  protected
    procedure RegisterIntrinsic; override;
  end;

  TQTXFileWriteCB = procedure (TagValue: variant; Error: Exception);
  TQTXFileStateCB = procedure (TagValue: variant; Error: Exception);

  TQTXUnRegisterLocalDeviceCB = procedure (TagValue: variant; DiskName: string; Error: Exception);
  TQTXRegisterLocalDeviceCB = procedure (TagValue: variant; LocalPath: string; Error: Exception);
  TQTXFindDeviceCB = procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception);
  TQTXGetDisksCB = procedure (TagValue: variant; Devices: JDeviceList; Error: Exception);

  TQTXGetFileInfoCB = procedure (TagValue: variant; LocalName: string; Info: JStats; Error: Exception);
  TQTXGetTranslatePathCB = procedure (TagValue: variant; Original, Translated: string; Error: Exception);

  TQTXCheckDevicePathCB = procedure (TagValue: variant; PathName: string; Error: Exception);

  TQTXServerExecuteCB = procedure (TagValue: variant; Data: string; Error: Exception);

  TQTXTaskService = class(TRagnarokService)
  private
    FPrefs:     TQTXIniFile;
    FLog:       TQTXLogEmitter;
    FDatabase:  TSQLite3Database;

    FZConfig:   TQTXZConfigClient;
    FRegHandle: TQTXDispatchHandle;
    FRegCount:  integer;

    procedure   HandleGetDevices(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleGetDeviceByName(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleCreateLocalDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleDestroyDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleFileRead(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleFileReadPartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleGetFileInfo(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleFileDelete(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);

    procedure   HandleFileWrite(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleFileWritePartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleFileRename(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleGetDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);

    procedure   HandleMkDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
    procedure   HandleRmDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);

    procedure   ExecuteExternalJS(Params: array of string;
      TagValue: variant; const CB: TQTXServerExecuteCB);

    procedure   SendError(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage; Message: string);

  protected
    function    GetFactory: TMessageFactory; override;
    procedure   SetupPreferences(const CB: TRagnarokServiceCB);
    procedure   SetupLogfile(LogFileName: string;const CB: TRagnarokServiceCB);
    procedure   SetupDatabase(const CB: TRagnarokServiceCB);

    procedure   ValidateLocalDiskName(TagValue: variant; Username, DeviceName: string; CB: TQTXCheckDevicePathCB);
    procedure   RegisterLocalDevice(TagValue: variant; Username, DiskName: string; CB: TQTXRegisterLocalDeviceCB);
    procedure   UnRegisterLocalDevice(TagValue: variant; UserName, DiskName:string; CB: TQTXUnRegisterLocalDeviceCB);

    procedure   GetDevicesForUser(TagValue: variant; UserName: string; CB: TQTXGetDisksCB);
    procedure   FindDeviceByName(TagValue: variant; UserName, DiskName: string; CB: TQTXFindDeviceCB);
    procedure   FindDeviceByType(TagValue: variant; UserName: string; &Type: JDeviceType; CB: TQTXGetDisksCB);

    procedure   GetTranslatedPathFor(TagValue: variant; Username, FullPath: string; CB: TQTXGetTranslatePathCB);

    procedure   GetFileInfo(TagValue: variant; UserName: string; FullPath: string; CB: TQTXGetFileInfoCB);

    procedure   SetupTaskTable(const TagValue: variant; const CB: TRagnarokServiceCB);
    procedure   SetupOperationsTable(const TagValue: variant; const CB: TRagnarokServiceCB);
    procedure   SetupDeviceTable(const TagValue: variant; const CB: TRagnarokServiceCB);

    procedure   AfterServerStarted; override;
    procedure   BeforeServerStopped; override;
    procedure   Dispatch(Socket: TNJWebSocketSocket; Message: TQTXBaseMessage); override;

  public
    property    Preferences: TQTXIniFile read FPrefs;
    property    Database: TSQLite3Database read FDatabase;

    procedure   SetupService(const CB: TRagnarokServiceCB);

    constructor Create; override;
    destructor  Destroy; override;
  end;


implementation

//#############################################################################
// TQTXFileenticationFactory
//#############################################################################

procedure TQTXTaskServiceFactory.RegisterIntrinsic;
begin
  writeln("Registering task interface");
  &Register(TQTXFileGetDeviceListRequest);
  &Register(TQTXFileGetDeviceByNameRequest);
  &Register(TQTXFileCreateLocalDeviceRequest);
  &Register(TQTXFileDestroyDeviceRequest);
  &Register(TQTXFileReadPartialRequest);
  &Register(TQTXFileReadRequest);
  &Register(TQTXFileWritePartialRequest);
  &Register(TQTXFileWriteRequest);
  &Register(TQTXFileDeleteRequest);
  &Register(TQTXFileRenameRequest);
  &Register(TQTXFileInfoRequest);
  &Register(TQTXFileDirRequest);
  &Register(TQTXMkDirRequest);
  &Register(TQTXRmDirRequest);
  &Register(TQTXFileRenameRequest);
  &Register(TQTXFileDirRequest);
end;

//#############################################################################
// TQTXTaskService
//#############################################################################

constructor TQTXTaskService.Create;
begin
  inherited Create;
  FPrefs := TQTXIniFile.Create();
  FLog := TQTXLogEmitter.Create();
  FDatabase := TSQLite3Database.Create(nil);

  FZConfig := TQTXZConfigClient.Create();
  FZConfig.Port := 2292;

  self.OnUserSignedOff := procedure (Sender: TObject; Username: string)
  begin
    WriteToLogF("We got a service signal! User [%s] has signed off completely", [Username]);
  end;

  MessageDispatch.RegisterMessage(TQTXFileGetDeviceListRequest, @HandleGetDevices);
  MessageDispatch.RegisterMessage(TQTXFileGetDeviceByNameRequest, @HandleGetDeviceByName);
  MessageDispatch.RegisterMessage(TQTXFileCreateLocalDeviceRequest, @HandleCreateLocalDevice);
  MessageDispatch.RegisterMessage(TQTXFileDestroyDeviceRequest, @HandleDestroyDevice);

  MessageDispatch.RegisterMessage(TQTXFileReadRequest, @HandleFileRead);
  MessageDispatch.RegisterMessage(TQTXFileReadPartialRequest, @HandleFileReadPartial);

  MessageDispatch.RegisterMessage(TQTXFileWriteRequest, @HandleFileWrite);
  MessageDispatch.RegisterMessage(TQTXFileWritePartialRequest, @HandleFileWritePartial);

  MessageDispatch.RegisterMessage(TQTXFileInfoRequest, @HandleGetFileInfo);
  MessageDispatch.RegisterMessage(TQTXFileDeleteRequest, @HandleFileDelete);

  MessageDispatch.RegisterMessage(TQTXMkDirRequest, @HandleMkDir);
  MessageDispatch.RegisterMessage(TQTXRmDirRequest, @HandleRmDir);
  MessageDispatch.RegisterMessage(TQTXFileRenameRequest, @HandleFileRename);

  MessageDispatch.RegisterMessage(TQTXFileDirRequest, @HandleGetDir);
end;

destructor TQTXTaskService.Destroy;
begin
  // decouple logger from our instance
  self.logging := nil;

  // Release prefs + log
  FPrefs.free;
  FLog.free;
  FZConfig.free;
  inherited;
end;

procedure TQTXTaskService.SendError(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage; Message: string);
begin
  var reply := TQTXErrorMessage.Create(request.ticket);
  try
    reply.Code := CNT_MESSAGE_CODE_ERROR;
    reply.Routing.TagValue := Request.Routing.TagValue;
    reply.Response := Message;

    if Socket.ReadyState = rsOpen then
    begin
      try
        Socket.Send( reply.Serialize() );
      except
        on e: exception do
        WriteToLog(e.message);
      end;
    end else
      WriteToLog("Failed to dispatch error, socket is closed error");
  finally
    reply.free;
  end;
end;

procedure TQTXTaskService.ExecuteExternalJS(Params: array of string;
  TagValue: variant; const CB: TQTXServerExecuteCB);
begin
  var LTask: JChildProcess;

  var lOpts := TVariant.CreateObject();
  lOpts.shell := false;
  lOpts.detached := true;

  Params.insert(0, '--no-warnings');

  // Spawn a new process, this creates a new shell interface
  try
    LTask := child_process().spawn('node', Params, lOpts );
  except
    on e: exception do
    begin
      if assigned(CB) then
        CB(TagValue, e.message, e);
      exit;
    end;
  end;

  // Map general errors on process level
  LTask.on('error', procedure (error: variant)
  begin
    {$IFDEF DEBUG}
    writeln("error->" + error.toString());
    {$ENDIF}
    WriteToLog(error.toString());

    if assigned(CB) then
      CB(TagValue, "", nil);
  end);

  // map stdout so we capture the output
  LTask.stdout.on('data', procedure (data: variant)
  begin
    if assigned(CB) then
      CB(TagValue, data.toString(), nil);
  end);

  // map stderr so we can capture exception messages
  LTask.stderr.on('data', procedure (error:variant)
  begin
    {$IFDEF DEBUG}
    writeln("stdErr->" + error.toString());
    {$ENDIF}

    if assigned(CB) then
      CB(TagValue, "", nil);

    WriteToLog(error.toString());
  end);
end;

function TQTXTaskService.GetFactory: TMessageFactory;
begin
  result := TQTXTaskServiceFactory.Create();
end;

procedure TQTXTaskService.SetupService(const CB: TRagnarokServiceCB);
begin
  SetupPreferences( procedure (Error: Exception)
  begin
    // No logfile yet setup (!)
    if Error  nil then
    begin
      WriteToLog("Preferences setup: Failed!");
      WriteToLog(error.message);
      raise error;
    end else
    WriteToLog("Preferences setup: OK");

    // logfile-name is always relative to the executable
    var LLogFileName := TQTXNodeFileUtils.IncludeTrailingPathDelimiter( TQTXNodeFileUtils.GetCurrentDirectory );
    LLogFileName += FPrefs.ReadString('log', 'logfile', 'log.txt');

    // Port is defined in the ancestor, so we assigns it here
    Port := FPrefs.ReadInteger('networking', 'port', CNT_PREFS_DEFAULTPORT);

    SetupLogfile(LLogFileName, procedure (Error: Exception)
    begin
      if Error  nil then
      begin
        WriteToLog("Logfile setup: Failed!");
        WriteToLog(error.message);
        raise error;
      end else
      WriteToLog("Logfile setup: OK");

      SetupDatabase( procedure (Error: Exception)
      begin
        if Error  nil then
        begin
          WriteToLog("Database setup: Failed!");
          WriteToLog(error.message);
          if assigned(CB) then
            CB(Error)
          else
            raise Error;
        end else
        WriteToLog("Database setup: OK");

        if assigned(CB) then
          CB(nil);
      end);

    end);
  end);
end;

procedure TQTXTaskService.SetupPreferences(const CB: TRagnarokServiceCB);
begin
  var lBasePath := TQTXNodeFileUtils.GetCurrentDirectory;
  var LPrefsFile := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + CNT_PREFS_FILENAME;

  if TQTXNodeFileUtils.FileExists(LPrefsFile) then
  begin
    FPrefs.LoadFromFile(nil, LPrefsFile, procedure (TagValue: variant; Error: Exception)
    begin
      if Error  nil then
      begin
        if assigned(CB) then
          CB(Error)
        else
          raise Error;
        exit;
      end;

      if assigned(CB) then
        CB(nil);
    end);

  end else
  begin
    var LError := Exception.Create('Could not locate preferences file: ' + LPrefsFile);
    WriteToLog(LError.message);
    if assigned(CB) then
      CB(LError)
    else
      raise LError;
  end;
end;

procedure TQTXTaskService.SetupLogfile(LogFileName: string;const CB: TRagnarokServiceCB);
begin
  // Attempt to open logfile
  // Note: Log-object error options is set to throw exceptions
  try
    FLog.Open(LogFileName);
  except
    on e: exception do
    begin
      if assigned(CB) then
      begin
        CB(e);
        exit;
      end else
      begin
        writeln(e.message);
        raise;
      end;
    end;
  end;

  // We inherit from TQTXLogObject, which means we can pipe
  // all errors etc directly using built-in functions. So here
  // we simply glue our instance to the log-file, and its all good
  self.Logging := FLog as IQTXLogClient;

  if assigned(CB) then
    CB(nil);
end;

procedure TQTXTaskService.FindDeviceByType(TagValue: variant; UserName: string; &Type: JDeviceType; CB: TQTXGetDisksCB);
begin
  UserName := username.trim().ToLower();
  if Username.length < 1 then
  begin
    WriteToLog("Failed to lookup disk, username was invalid error");
    var lError := EException.Create("Failed to lookup devices, invalid username");
    if assigned(CB) then
      CB(TagValue, nil, lError)
    else
      raise lError;
    exit;
  end;

  GetDevicesForUser(TagValue, Username,
  procedure (TagValue: variant; Devices: JDeviceList; Error: Exception)
  begin
    if Error  nil then
    begin
      if assigned(CB) then
        CB(TagValue, nil, Error)
      else
        raise Error;
      exit;
    end;

    var x := 0;
    while x < Devices.dlDrives.Count do
    begin
      if Devices.dlDrives[x].&Type  &Type then
      begin
        Devices.dlDrives.delete(x, 1);
        continue;
      end;
      inc(x);
    end;

    if assigned(CB) then
      CB(TagValue, Devices, nil);
  end);
end;

procedure TQTXTaskService.FindDeviceByName(TagValue: variant; Username, DiskName: string; CB: TQTXFindDeviceCB);
begin
  UserName := username.trim().ToLower();
  if Username.length < 1 then
  begin
    var lLogText := "Failed to lookup device, username was invalid error";
    WriteToLog(lLogText);
    var lError := EException.Create(lLogText);
    if assigned(CB) then
      CB(TagValue, nil, lError)
    else
      raise lError;
    exit;
  end;

  DiskName := DiskName.trim();
  if DiskName.length < 1 then
  begin
    var lLogText := "Failed to lookup device, disk-name was invalid error";
    WriteToLog(lLogText);
    var lError := EException.Create(lLogText);
    if assigned(CB) then
      CB(TagValue, nil, lError)
    else
      raise lError;
    exit;
  end;

  GetDevicesForUser(TagValue, Username,
  procedure (TagValue: variant; Devices: JDeviceList; Error: Exception)
  begin
    if Error  nil then
    begin
      if assigned(CB) then
        CB(TagValue, nil, Error)
      else
        raise Error;
      exit;
    end;

    DiskName := DiskName.trim().ToLower();
    var lDiskInfo: JDeviceInfo := nil;


    for var disk in Devices.dlDrives do
    begin
      if disk.Name.ToLower() = DiskName then
      begin
        lDiskInfo := disk;
        break;
      end;
    end;

    if assigned(CB) then
      CB(TagValue, lDiskInfo, nil);
  end);
end;

procedure TQTXTaskService.GetTranslatedPathFor(TagValue: variant; Username, FullPath: string; CB: TQTXGetTranslatePathCB);
begin
  var lParser := TQTXPathParser.Create();
  try
    var lInfo: TQTXPathData;
    if lparser.Parse(FullPath, lInfo) then
    begin
      // Locate the device for the path belonging to the user
      FindDeviceByName(TagValue, UserName, lInfo.MountPart,
      procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
      begin
        if Error  nil then
        begin
          if assigned(CB) then
            CB(TagValue, FullPath, '', Error)
          else
            raise Error;
          exit;
        end;

        if Device.&Type  dtLocal then
        begin
          var lError := EException.CreateFmt('Failed to translate path, device [%s] is not local error', [Device.Name]);
          if assigned(CB) then
            CB(TagValue, FullPath, '', Error)
          else
            raise Error;
          exit;
        end;

        // We want the path + filename, so we can append that to
        // the actual localized filesystem
        var lExtract := FullPath;
        delete(lExtract, 1, lInfo.MountPart.Length + 1);

        // Construct complete storage location
        var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
        lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
        lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
        lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;

        // Return translated path
        if assigned(CB) then
          CB(TagValue, FullPath, lFullPath, nil);

      end);
    end else
    begin
      var lErr := EException.CreateFmt("Invalid path [%s] error", [FullPath]);
      if assigned(CB) then
        CB(TagValue, FullPath, '', lErr)
      else
        raise lErr;
    end;
  finally
    lParser.free;
  end;
end;

procedure TQTXTaskService.GetFileInfo(TagValue: variant; UserName, FullPath: string; CB: TQTXGetFileInfoCB);
begin
  var lParser := TQTXPathParser.Create();
  try
    var lInfo: TQTXPathData;
    if lparser.Parse(FullPath, lInfo) then
    begin
      // Locate the device for the path belonging to the user
      FindDeviceByName(TagValue, UserName, lInfo.MountPart,
      procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
      begin
        if Error  nil then
        begin
          if assigned(CB) then
            CB(TagValue, '', nil, Error)
          else
            raise Error;
          exit;
        end;

        case Device.&Type of
        dtLocal:
          begin
            // We want the path + filename, so we can append that to
            // the actual localized filesystem
            var lExtract := FullPath;
            delete(lExtract, 1, lInfo.MountPart.Length + 1);

            // Construct complete storage location
            var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;

            // Call the underlying OS to get the file statistics
            NodeJsFsAPI().lStat(lFullPath,
            procedure (Error: JError; Stats: JStats)
            begin
              if Error  nil then
              begin
                var lError := EException.Create(Error.message);
                if assigned(CB) then
                  CB(TagValue, lFullPath, nil, lError)
                else
                  raise lError;
                exit;
              end;

              // And deliver
              if assigned(CB) then
                CB(TagValue, lFullPath, Stats, nil);
            end);
          end;
        dtDropbox, dtGoogle, dtMsDrive:
          begin
            var lError := EException.Create("Cloud bindings not activated error");
            if assigned(CB) then
              CB(TagValue, '', nil, lError)
          end;
        end;
      end);
    end else
    begin
      var lErr := EException.CreateFmt("Invalid path [%s] error", [FullPath]);
      if assigned(CB) then
        CB(TagValue, '', nil, lErr)
      else
        raise lErr;
    end;
  finally
    lParser.free;
  end;
end;

procedure TQTXTaskService.GetDevicesForUser(TagValue: variant; Username: string; CB: TQTXGetDisksCB);
begin
  UserName := username.trim().ToLower();
  if Username.length < 1 then
  begin
    WriteToLog("Failed to lookup devices, username was invalid error");
    var lError := EException.Create("Failed to lookup devices, invalid username");
    if assigned(CB) then
      CB(TagValue, nil, lError)
    else
      raise lError;
    exit;
  end;

  var lTransaction: TQTXReadTransaction;
  if not TSQLite3Database(DataBase).CreateReadTransaction(lTransaction) then
  begin
    var lErr := EException.Create("Failed to create read-transaction error");
    if assigned(cb) then
      CB(TagValue, nil, lErr)
    else
      raise lErr;
    exit;
  end;

  var lQuery := TSQLite3ReadTransaction(lTransaction);
  lQuery.SQL := "select * from devices where owner=?";
  lQuery.Parameters.AddValueOnly(Username);

  lQuery.Execute(
  procedure (Sender: TObject; Error: Exception)
  begin
    if Error  nil then
    begin
      if assigned(CB) then
        CB(TagValue, nil, Error)
      else
        raise Error;
      exit;
    end;

    var lDisks := new JDeviceList();
    lDisks.dlUser := UserName;

    for var x := 0 to lQuery.datarows.length-1 do
    begin
      var lInfo := new JDeviceInfo();
      lInfo.Name := lQuery.datarows[x]["name"];
      lInfo.&Type := JDeviceType( lQuery.datarows[x]["type"] );
      lInfo.owner := lQuery.datarows[x]["owner"];
      lInfo.location := lQuery.datarows[x]["location"];
      lInfo.APIKey := lQuery.datarows[x]["apikey"];
      lInfo.APISecret := lQuery.datarows[x]["apisecret"];
      lInfo.APIPassword := lQuery.datarows[x]["apipassword"];
      lInfo.APIUser := lQuery.datarows[x]["apiuser"];
      lDisks.dlDrives.add(lInfo);
    end;

    try
      if assigned(CB) then
        CB(TagValue, lDisks, nil);
    finally
      lQuery.free;
    end;
  end);
end;

procedure TQTXTaskService.ValidateLocalDiskName(TagValue: variant; Username, DeviceName: string; CB: TQTXCheckDevicePathCB);
begin
  var Filename := 'disk.' + username + '.' + DeviceName + '.' + ord(JDeviceType.dtLocal).ToString();

  var LBasePath := TQTXNodeFileUtils.GetCurrentDirectory();
  LBasePath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + 'userdevices';

  // Make sure the device folder is there
  if not TQTXNodeFileUtils.DirectoryExists(LBasePath) then
  begin
    var lError := EException.CreateFmt("Directory not found: %s", [lBasePath]);
    if assigned(CB) then
      CB(TagValue, '', lError)
    else
      raise lError;
    exit;
  end;

  lBasePath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + Filename;

  if TQTXNodeFileUtils.DirectoryExists(LBasePath) then
  begin
    var lError := EException.CreateFmt("Path already exist error [%s]", [lBasePath]);
    if assigned(CB) then
      CB(TagValue, '', lError)
    else
      raise lError;
    exit;
  end;

  // OK, folder is not created yet, so its good to go
  if assigned(CB) then
    CB(TagValue, Filename, nil);
end;

procedure TQTXTaskService.UnRegisterLocalDevice(TagValue: variant; UserName, DiskName: string; CB: TQTXUnRegisterLocalDeviceCB);
begin
  WriteToLogF("Removing local device [%s] for user [%s] ", [DiskName, Username]);

  // Check username string
  UserName := username.trim().ToLower();
  if Username.length < 1 then
  begin
    WriteToLog("Failed to unregister device, username was invalid error");
    var lError := EException.Create("Failed to register device, invalid username");
    if assigned(CB) then
      CB(TagValue, DiskName, lError)
    else
      raise lError;
    exit;
  end;

  // Check diskname string
  DiskName := DiskName.trim().ToLower();
  if DiskName.length < 1 then
  begin
    WriteToLog("Failed to unregister device, disk-name was invalid error");
    var lError := EException.Create("Failed to register device, invalid disk-name");
    if assigned(CB) then
      CB(TagValue, DiskName, lError)
    else
      raise lError;
    exit;
  end;

  FindDeviceByName(TagValue, Username, DiskName,
  procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
  begin
    // Did the search fail?
    if Error  nil then
    begin
      WriteToLog(Error.message);
      if assigned(CB) then
        CB(TagValue, DiskName, Error)
      else
        raise Error;
      exit;
    end;

    // Make sure the device is local
    if Device.&Type  dtLocal then
    begin
      var lError := EException.CreateFmt('Failed to translate path, device [%s] is not local error', [Device.Name]);
      if assigned(CB) then
        CB(TagValue, DiskName, Error)
      else
        raise Error;
      exit;
    end;

    // Delete record from database
    var lWriter: TQTXWriteTransaction;
    if FDatabase.CreateWriteTransaction(lWriter) then
    begin
      lWriter.SQL := "delete from profiles where user = ? and name = ?;";
      lWriter.Parameters.AddValueOnly(Username);
      lWriter.Parameters.AddValueOnly(DiskName);

      lWriter.Execute(
      procedure (Sender: TObject; Error: Exception)
      begin
        try

          if Error  nil then
          begin
            if assigned(CB) then
              CB(TagValue, DiskName, Error)
            else
              raise Error;
            exit;
          end;

          // Construct complete storage location
          var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
          lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
          lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();

          // Now delete the disk-drive directory
          TQTXNodeFileUtils.DeleteDirectory(nil, lFullPath,
          procedure (TagValue: variant; Path: string; Error: Exception)
          begin
            if assigned(CB) then
              CB(TagValue, DiskName, Error)
          end);

        finally
          lWriter.free;
          lWriter := nil;
        end;
      end);
    end;
  end);
end;

procedure TQTXTaskService.RegisterLocalDevice(TagValue: variant; Username, DiskName: string; CB: TQTXRegisterLocalDeviceCB);
begin
  WriteToLogF("Adding local device [%s] for user [%s] ", [DiskName, Username]);

  UserName := username.trim().ToLower();
  if Username.length < 1 then
  begin
    WriteToLog("Failed to register device, username was invalid error");
    var lError := EException.Create("Failed to register device, invalid username");
    if assigned(CB) then
      CB(TagValue, '', lError)
    else
      raise lError;
    exit;
  end;

  DiskName := DiskName.trim().ToLower();
  if DiskName.length < 1 then
  begin
    WriteToLog("Failed to register device, disk-name was invalid error");
    var lError := EException.Create("Failed to register device, invalid disk-name");
    if assigned(CB) then
      CB(TagValue, '', lError)
    else
      raise lError;
    exit;
  end;

  FindDeviceByName(TagValue, Username, DiskName,
  procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
  begin
    // Did the search fail?
    if Error  nil then
    begin
      WriteToLog(Error.message);
      if assigned(CB) then
        CB(TagValue, '', Error)
      else
        raise Error;
      exit;
    end;

    // Does a device that match already exist?
    if Device  nil then
    begin
      var lError := EException.CreateFmt("Failed to create device [%s], device already exists", [DiskName]);
      if assigned(CB) then
        CB(TagValue, '', lError)
      else
        raise lError;
      exit;
    end;

    //  make sure the device-folder does not exist, so we can create it
    ValidateLocalDiskName(TagValue, Username, DiskName,
    procedure (TagValue: variant; PathName: string; Error: Exception)
    begin
      if Error  nil then
      begin
        if assigned(CB) then
          CB(TagValue, '', Error)
        else
          raise Error;
        exit;
      end;

      // ValidateLocalDiskName only returns the valid directory-name,
      // not a full path -- so we need to build up the full targetpath
      var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
      lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
      lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + PathName;

      TQTXNodeFileUtils.CreateDirectory(nil, lFullPath,
      procedure (TagValue: variant; Path: string; Error: exception)
      begin
        if Error  nil then
        begin
          var lError := EException.CreateFmt("Failed to create device [%s] with path: %s", [DiskName, lFullPath]);
          if assigned(CB) then
            CB(TagValue, PathName, lError)
          else
            raise lError;
          exit;
        end;

        FDatabase.Execute(
          #'insert into devices (type, owner, name, location)
            values(?, ?, ?, ?);',
            [ord(JDeviceType.dtLocal), UserName, Diskname, PathName] ,
        procedure (Sender: TObject; Error: Exception)
        begin
          if Error  nil then
          begin
            WriteToLog(Error.message);
            if assigned(CB) then
              CB(TagValue, PathName, Error)
            else
              raise Error;
            exit;
          end;

          WriteToLogF("Device [%s] added to database user [%s]", [DiskName, UserName]);
          if assigned(CB) then
            CB(TagValue, PathName, nil);
        end);

      end);



    end);
  end);
end;

procedure TQTXTaskService.SetupDeviceTable(const TagValue: variant; const CB: TRagnarokServiceCB);
begin

  FDatabase.Execute(
    #'
      create table if not exists devices
          (
            id integer primary key AUTOINCREMENT,
            type        integer,
            owner       text,
            name        text,
            location    text,
            apikey      text,
            apisecret   text,
            apipassword text,
            apiuser     text
          );
          ', [],
    procedure (Sender: TObject; Error: Exception)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.message);
        if assigned(CB) then
          CB(Error)
        else
          raise Error;
        exit;
      end else
      if assigned(CB) then
        CB(nil);
    end);
end;

procedure TQTXTaskService.SetupTaskTable(const TagValue: variant; const CB: TRagnarokServiceCB);
begin

  FDatabase.Execute(
    #'
      create table if not exists tasks
          (
            id integer primary key AUTOINCREMENT,
            state     integer,
            username  text,
            created   real
          );
          ', [],
    procedure (Sender: TObject; Error: Exception)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.message);
        if assigned(CB) then
          CB(Error)
        else
          raise Error;
        exit;
      end else
      if assigned(CB) then
        CB(nil);
    end);
end;


procedure TQTXTaskService.SetupOperationsTable(const TagValue: variant; const CB: TRagnarokServiceCB);
begin
  FDatabase.Execute(
    #'
      create table if not exists operations
          (
            id integer primary key AUTOINCREMENT,
            username text,
            taskid integer,
            name text,
            filename text
          );
          ', [],
    procedure (Sender: TObject; Error: Exception)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.message);
        if assigned(CB) then
          CB(Error)
        else
          raise Error;
        exit;
      end else
      if assigned(CB) then
        CB(nil);
    end);
end;

procedure TQTXTaskService.SetupDatabase(const CB: TRagnarokServiceCB);
begin
  // Try to read database-path from preferences file
  var LDbFileToOpen := FPrefs.ReadString("database", "database_name", "");

  // Trim away spaces, check if there is a filename
  LDbFileToOpen := LDbFileToOpen.trim();
  if LDbFileToOpen.length < 1 then
  begin
    // No filename? Fall back on pre-defined file in CWD
    var LBasePath := TQTXNodeFileUtils.GetCurrentDirectory();
    LDbFileToOpen := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + CNT_PREFS_DBNAME;
  end;

  FDatabase.AccessMode := TSQLite3AccessMode.sqaReadWriteCreate;
  FDatabase.Open(LDbFileToOpen,
    procedure (Sender: TObject; Error: Exception)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.message);
        if assigned(CB) then
          CB(Error)
        else
          raise Error;
        exit;
      end;

      WriteToLog("Initializing task table");
      SetupTaskTable(nil, procedure (Error: exception)
      begin
        if Error  nil then
        begin
          WriteToLog("Tasks initialized: **failed");
          WriteToLog(error.message);
          if assigned(CB) then
            CB(Error)
          else
            raise Error;
          exit;
        end else
        writeToLog("Tasks initialized: OK");

        WriteToLog("Initializing operations table");
        SetupOperationsTable(nil, procedure (Error: exception)
        begin
          if Error  nil then
          begin
            WriteToLog("Operations initialized: **failed");
            WriteToLog(error.message);
            if assigned(CB) then
              CB(Error);
            exit;
          end else
          writeToLog("Operations initialized: OK");

          WriteToLog("Initializing device table");
          SetupDeviceTable(nil, procedure (Error: exception)
          begin
            if Error  nil then
            begin
              WriteToLog("Device-table initialized: **failed");
              WriteToLog(error.message);
              if assigned(CB) then
                CB(Error);
              exit;
            end else
            writeToLog("Device-table initialized: OK");

            if assigned(CB) then
              CB(nil);
          end);
        end);
      end);
    end);
end;


procedure TQTXTaskService.HandleFileRead(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXFileReadRequest(request);
  var lUserName := lRequest.UserName;
  var lFileName := lRequest.FileName;

  // Check filename length
  if lFileName.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  GetFileInfo(lRequest, lUserName, lFileName,
  procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    var lOptions: TReadFileOptions;
    lOptions.encoding := 'binary';

    NodeJsFsAPI().readFile(LocalFile, lOptions,
    procedure (Error: JError; Data: JNodeBuffer)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.message);
        SendError(Socket, Request, Error.Message);
        exit;
      end;

      var lResponse := TQTXFileReadResponse.Create(Request.Ticket);
      lResponse.UserName := lUserName;
      lResponse.Routing.TagValue := request.routing.tagValue;
      lResponse.FileName := lFileName;
      lResponse.Code := CNT_MESSAGE_CODE_OK;
      lResponse.Response := CNT_MESSAGE_TEXT_OK;

      // Convert filedata in one pass
      try
        var lConvert := TDataTypeConverter.Create();
        try
          lResponse.Attachment.AppendBytes( lConvert.TypedArrayToBytes(Data) );
        finally
          lConvert.free;
        end;
      except
        on e: exception do
        begin
          WriteToLog(e.message);
          SendError(Socket, Request, e.Message);
          exit;
        end;
      end;

      try
        Socket.Send( lResponse.Serialize() );
      except
        on e: exception do
          WriteToLog(e.message);
      end;
    end);
  end);
end;

procedure TQTXTaskService.HandleFileReadPartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXFileReadPartialRequest(request);
  var lUserName := lRequest.UserName;
  var lFileName := lRequest.FileName;
  var lStart := lRequest.Offset;
  var lSize := lRequest.Size;

  // Check filename length
  if lFileName.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  if lSize < 1 then
  begin
    SendError(Socket, Request, "Read failed, invalid size error");
    exit;
  end;

  if lStart < 0 then
  begin
    SendError(Socket, Request, "Read failed, invalid offset error");
    exit;
  end;

  GetFileInfo(lRequest, lUserName, lFileName,
  procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    if lStart > Info.size then
    begin
      SendError(Socket, Request, "Read failed, offset beyond filesize error");
      exit;
    end;

    NodeJsFsAPI().open(LocalFile, "r",
    procedure (Error: JError; Fd: THandle)
    begin
      if error  nil then
      begin
        WriteToLog(Error.message);
        SendError(Socket, Request, Error.Message);
        exit;
      end;

      var Data = new JNodeBuffer(lSize);
      NodeJsFsAPI().read(Fd, Data, 0, lSize, lStart,
      procedure (Error: JError; BytesRead: integer; buffer: JNodeBuffer)
      begin
        if Error  nil then
        begin
          NodeJsFsAPI().closeSync(Fd);
          WriteToLog(Error.message);
          SendError(Socket, Request, Error.Message);
          exit;
        end;

        // Close the file-handle and return data
        NodeJsFsAPI().close(Fd, procedure (Error: JError)
        begin
          var lResponse := TQTXFileReadPartialResponse.Create(Request.Ticket);
          lResponse.UserName := lUserName;
          lResponse.Routing.TagValue := request.routing.tagValue;
          lResponse.FileName := lFileName;
          lResponse.Code := CNT_MESSAGE_CODE_OK;
          lResponse.Response := CNT_MESSAGE_TEXT_OK;

          // Only encode data if read
          if BytesRead > 0 then
          begin
            // Convert filedata in one pass
            try
              var lConvert := TDataTypeConverter.Create();
              try
                lResponse.Attachment.AppendBytes( lConvert.TypedArrayToBytes(buffer) );
              finally
                lConvert.free;
              end;
            except
              on e: exception do
              begin
                WriteToLog(e.message);
                SendError(Socket, Request, e.Message);
                exit;
              end;
            end;
          end;

          try
            Socket.Send( lResponse.Serialize() );
          except
            on e: exception do
              WriteToLog(e.message);
          end;

        end);
      end);
    end);
  end);
end;

procedure TQTXTaskService.HandleFileWrite(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest  := TQTXFileWriteRequest(request);
  var lFileName := lRequest.FileName.trim();
  var lUserName := lRequest.UserName.trim();

  var FullPath  := lFileName;

  // Check filename length
  if lFileName.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  var lParser := TQTXPathParser.Create();
  try
    var lInfo: TQTXPathData;
    if lparser.Parse(FullPath, lInfo) then
    begin
      // Locate the device for the path belonging to the user
      FindDeviceByName(nil, lUserName, lInfo.MountPart,
      procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
      begin
        if Error  nil then
        begin
          WriteToLog(Error.Message);
          SendError(Socket, Request, Error.Message);
          exit;
        end;

        case Device.&Type of
        dtLocal:
          begin
            // We want the path + filename, so we can append that to
            // the actual localized filesystem
            var lExtract := FullPath;
            delete(lExtract, 1, lInfo.MountPart.Length + 1);

            // Construct complete storage location
            var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;

            // Extract data to be appended, if any
            // note: null bytes should be allowed, it should just create the file
            var lBytes: array of UInt8;
            if lRequest.attachment.Size > 0 then
              lBytes := lRequest.Attachment.ToBytes();

            // Write the data to the file
            NodeJsFsAPI().writeFile(lFullPath, lBytes,
            procedure (Error: JError)
            begin
              if Error  nil then
              begin
                WriteToLog(Error.Message);
                SendError(Socket, Request, Error.Message);
                exit;
              end;

              // Setup response object
              var lResponse := TQTXFileWriteResponse.Create(lRequest.Ticket);
              lResponse.UserName := lUserName;
              lResponse.FileName := lFileName;
              lResponse.Code := CNT_MESSAGE_CODE_OK;
              lResponse.Response := CNT_MESSAGE_TEXT_OK;

              // Send success response
              try
                Socket.Send( lResponse.Serialize() );
              except
                on e: exception do
                  WriteToLog(e.message);
              end;

            end);

          end;
        dtDropbox, dtGoogle, dtMsDrive:
          begin
            var lErrorText := Format("Clound bindings not active error [%s]", [lRequest.FileName]);
            WriteToLog(lErrorText);
            SendError(Socket, Request, lErrorText);
          end;
        end;
      end);
    end else
    begin
      SendError(Socket, Request, format("Invalid path [%s] error", [FullPath]));
    end;
  finally
    lParser.free;
  end;
end;

procedure TQTXTaskService.HandleFileWritePartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest  := TQTXFileWritePartialRequest(request);
  var lFileName  := lRequest.FileName.trim();
  var lUserName := lRequest.UserName.trim();
  var lFileOffset := lRequest.Offset;

  // Check filename length
  if lFileName.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  var FullPath := lFileName;

  var lParser := TQTXPathParser.Create();
  try
    var lInfo: TQTXPathData;
    if lparser.Parse(FullPath, lInfo) then
    begin
      // Locate the device for the path belonging to the user
      FindDeviceByName(nil, lUserName, lInfo.MountPart,
      procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
      begin
        if Error  nil then
        begin
          WriteToLog(Error.Message);
          SendError(Socket, Request, Error.Message);
          exit;
        end;

        case Device.&Type of
        dtLocal:
          begin
            // We want the path + filename, so we can append that to
            // the actual localized filesystem
            var lExtract := FullPath;
            delete(lExtract, 1, lInfo.MountPart.Length + 1);

            // Construct complete storage location
            var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
            lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;

            // Extract data to be appended, if any
            // note: null bytes should be allowed, it should just create the file
            var lBytes: array of UInt8;
            if lRequest.attachment.Size > 0 then
              lBytes := lRequest.Attachment.ToBytes();

            var lAccess := TQTXNodeFile.Create();
            lAccess.Open(lFullPath, TQTXNodeFileMode.nfWrite,
            procedure (Error: Exception)
            begin
              if Error  nil then
              begin
                WriteToLog(Error.Message);
                SendError(Socket, Request, Error.Message);
                exit;
              end;

              lAccess.Write(lBytes, lFileOffset,
              procedure (Error: Exception)
              begin
                if Error  nil then
                begin
                  WriteToLog(Error.Message);
                  SendError(Socket, Request, Error.Message);
                  exit;
                end;

                // Setup response object
                var lResponse := TQTXFileWriteResponse.Create(lRequest.Ticket);
                lResponse.UserName := lUserName;
                lResponse.FileName := lFileName;
                lResponse.Code := CNT_MESSAGE_CODE_OK;
                lResponse.Response := CNT_MESSAGE_TEXT_OK;

                // Send success response
                try
                  Socket.Send( lResponse.Serialize() );
                except
                  on e: exception do
                    WriteToLog(e.message);
                end;

              end);
            end);
          end;
        dtDropbox, dtGoogle, dtMsDrive:
          begin
            var lErrorText := Format("Clound bindings not active error [%s]", [lRequest.FileName]);
            WriteToLog(lErrorText);
            SendError(Socket, Request, lErrorText);
          end;
        end;
      end);
    end else
    begin
      SendError(Socket, Request, format("Invalid path [%s] error", [FullPath]));
    end;
  finally
    lParser.free;
  end;
end;

procedure TQTXTaskService.HandleRmDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXRmDirRequest(request);
  var lUserName := lRequest.UserName.trim();
  var lDirPath := lRequest.DirPath.trim();

  if lDirPath.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lDirPath) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  var lParser := TQTXPathParser.Create();
  try
    var lInfo: TQTXPathData;
    if lParser.Parse(lDirPath, lInfo) then
    begin
      GetTranslatedPathFor(nil, lUserName, lDirPath,
      procedure (TagValue: variant; Original, Translated: string; Error: Exception)
      begin
        if Error  nil then
        begin
          WriteToLog(Error.message);
          SendError(Socket, Request, Error.Message);
          exit;
        end;

        if not TQTXNodeFileUtils.DirectoryExists(Translated) then
        begin
          WriteToLogF("RmDir Failed, directory [%s] does not exist", [Translated]);
          SendError(Socket, Request, Format("RmDir failed, directory [%s] does not exist", [Original]));
          exit;
        end;

        TQTXNodeFileUtils.DeleteDirectory(nil, Translated,
        procedure (TagValue: variant; Path: string; Error: Exception)
        begin
          if error  nil then
          begin
            WriteToLog(Error.message);
            SendError(Socket, Request, Error.Message);
            exit;
          end;

          // Setup response object
          var lResponse := TQTXRmDirResponse.Create(lRequest.Ticket);
          lResponse.UserName := lUserName;
          lResponse.DirPath := lDirPath;
          lResponse.Code := CNT_MESSAGE_CODE_OK;
          lResponse.Response := CNT_MESSAGE_TEXT_OK;
          lResponse.Routing.TagValue := lRequest.Routing.TagValue;

          // Send success response
          try
            Socket.Send( lResponse.Serialize() );
          except
            on e: exception do
              WriteToLog(e.message);
          end;
        end);
      end);
    end else
    begin
      var lText := format("RmDir failed, invalid path [%s] error", [lDirPath]);
      WriteToLog(lText);
      SendError(Socket, Request, lText);
    end;
  finally
    lParser.free;
  end;
end;

procedure TQTXTaskService.HandleMkDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXMkDirRequest(request);
  var lUserName := lRequest.UserName.trim();
  var lDirPath := lRequest.DirPath.trim();

  if lDirPath.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lDirPath) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  var lParser := TQTXPathParser.Create();
  try
    var lInfo: TQTXPathData;
    if lparser.Parse(lDirPath, lInfo) then
    begin
      GetTranslatedPathFor(nil, lUserName, lDirPath,
      procedure (TagValue: variant; Original, Translated: string; Error: Exception)
      begin
        if Error  nil then
        begin
          WriteToLog(Error.message);
          SendError(Socket, Request, Error.Message);
          exit;
        end;

        TQTXNodeFileUtils.DirectoryExists(nil, Translated,
        procedure (TagValue: variant; Path: string; Error: Exception)
        begin
          if Error  nil then
          begin
            WriteToLogF("MkDir Failed, directory [%s] already exists", [Translated]);
            SendError(Socket, Request, Format("MkDir Failed, directory [%s] already exists", [Original]));
            exit;
          end;

          TQTXNodeFileUtils.CreateDirectory(nil, Translated,
          procedure (TagValue: variant; Path: string; Error: Exception)
          begin
            if Error  nil then
            begin
              WriteToLogF("MkDir Failed, directory [%s] could not be created", [Original]);
              SendError(Socket, Request, Format("MkDir Failed, directory [%s] could not be created", [Translated]));
              exit;
            end;

            // Setup response object
            var lResponse := TQTXMkDirResponse.Create(lRequest.Ticket);
            lResponse.UserName := lUserName;
            lResponse.DirPath := lDirPath;
            lResponse.Code := CNT_MESSAGE_CODE_OK;
            lResponse.Response := CNT_MESSAGE_TEXT_OK;
            lResponse.Routing.TagValue := lRequest.Routing.TagValue;

            // Send success response
            try
              Socket.Send( lResponse.Serialize() );
            except
              on e: exception do
                WriteToLog(e.message);
            end;

          end);
        end);
      end);

    end else
    begin
      var lText := format("MkDir Failed, invalid path [%s] error", [lDirPath]);
      WriteToLog(lText);
      SendError(Socket, Request, lText);
    end;
  finally
    lParser.free;
  end;
end;

procedure TQTXTaskService.HandleFileDelete(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXFileDeleteRequest(Request);
  var lUserName := lRequest.UserName.trim();
  var lFileName := lRequest.FileName.trim();

  if lFileName.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  GetFileInfo(lRequest, lUserName, lFileName,
  procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    if not Info.isFile then
    begin
      SendError(Socket, Request, "Filesystem object is not a file error");
      exit;
    end;

    NodeJsFsAPI().unlink(LocalFile,
    procedure (Error: JError)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.message);
        SendError(Socket, Request, Error.message);
        exit;
      end;

      var lResponse := new TQTXFileDeleteResponse(lRequest.Ticket);
      lResponse.Routing.TagValue := request.Routing.TagValue;
      lResponse.UserName := lUserName;
      lResponse.FileName := lFileName;
      lResponse.Code := CNT_MESSAGE_CODE_OK;
      lResponse.Response := CNT_MESSAGE_TEXT_OK;

      try
        Socket.Send( lResponse.Serialize() );
      except
        on e: exception do
          WriteToLog(e.message);
      end;
    end);
  end);
end;

procedure TQTXTaskService.HandleFileRename(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXFileRenameRequest(Request);
  var lUserName := lRequest.UserName.trim();
  var lFileName := lRequest.FileName.trim();
  var lNewName := lRequest.NewName.trim();

  // Check filename length
  if lFileName.length < 1 then
  begin
    SendError(Socket, Request, Format("Invalid or empty from-filename [%s] error", [lFileName]) );
    exit;
  end;

  // check newname length
  if lNewName.length  0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  if pos(lTemp, lNewName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  if pos(lTemp, lNewName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;


  GetFileInfo(lRequest, lUserName, lFileName,
  procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    if not Info.isFile then
    begin
      SendError(Socket, Request, "Filesystem object is not a file error");
      exit;
    end;

    GetTranslatedPathFor(nil, lUsername, lNewName,
    procedure (TagValue: variant; Original, Translated: string; Error: Exception)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.message);
        SendError(Socket, Request, Error.Message);
        exit;
      end;

      NodeJsFsAPI().rename(LocalFile, Translated,
      procedure (Error: JError)
      begin
        if Error  nil then
        begin
          WriteToLog(Error.message);
          SendError(Socket, Request, Error.message);
          exit;
        end;

        var lResponse := new TQTXFileRenameResponse(lRequest.Ticket);
        lResponse.Routing.TagValue := request.Routing.TagValue;
        lResponse.UserName := lUserName;
        lResponse.FileName := lFileName;
        lResponse.Code := CNT_MESSAGE_CODE_OK;
        lResponse.Response := CNT_MESSAGE_TEXT_OK;

        try
          Socket.Send( lResponse.Serialize() );
        except
          on e: exception do
            WriteToLog(e.message);
        end;
      end);

    end);

  end);
end;

procedure TQTXTaskService.HandleGetDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXFileDirRequest(Request);
  var lUserName := lRequest.UserName.trim();
  var lPath := lRequest.Path.trim();

  // prevent path escape attempts
  var lTemp := "../";
  if pos(lTemp, lPath) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lPath) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  GetTranslatedPathFor(nil, lUserName, lPath,
  procedure (TagValue: variant; Original, Translated: string; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    //writeln("Translated path is:" + Translated);

    if not TQTXNodeFileUtils.DirectoryExists(Translated) then
    begin
      WriteToLogF("GetDir Failed, directory [%s] does not exist", [Translated]);
      SendError(Socket, Request, Format("GetDir failed, directory [%s] does not exist", [Original]));
      exit;
    end;

    var lWalker := TQTXFileWalker.Create();
    lWalker.Examine(Translated, procedure (Sender: TQTXFileWalker; Error: EException)
    begin
      if Error  nil then
      begin
        WriteToLogF("GetDir Failed: %s", [Error.Message]);
        SendError(Socket, Request, Format("GetDir failed: %s", [Error.Message]));
        exit;
      end;

      // Get the directory data, swap out the path
      // record with the original [amiga] style path
      var lData := Sender.ExtractList();
      lData.dlPath := Original;

      var lResponse := new TQTXFileDirResponse(lRequest.Ticket);
      lResponse.Routing.TagValue := request.Routing.TagValue;
      lResponse.UserName := lUserName;
      lResponse.Path := lPath;
      lResponse.Assign( lData );

      try
        Socket.Send( lResponse.Serialize() );
      except
        on e: exception do
          WriteToLog(e.message);
      end;

      // release instance in 100ms
      TQTXDispatch.execute(procedure ()
      begin
        try
          lWalker.free
        except
          on e: exception do
          begin
            WriteToLogF("Failed to release file-walker instance: %s", [e.message]);
          end;
        end;
      end, 100);
    end);
  end);
end;

procedure TQTXTaskService.HandleGetFileInfo(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lRequest := TQTXFileInfoRequest(Request);
  var lUserName := lRequest.UserName.trim();
  var lFileName := lRequest.FileName.trim();

  // prevent path escape attempts
  var lTemp := "../";
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  lTemp := './';
  if pos(lTemp, lFileName) > 0 then
  begin
    SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
    exit;
  end;

  GetFileInfo(lRequest, lUserName, lFileName,
  procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    // Collect the data
    var lData := new JFileItem();
    lData.diFileName := lFileName;
    lData.diFileType := if Info.isFile then JFileItemType.wtFile else JFileItemType.wtFolder;
    lData.diFileSize := Info.size;
    lData.diFileMode := IntToStr(Info.mode);
    lData.diCreated  := TDateUtils.FromJsDate( Info.cTime );
    lData.diModified := TDateUtils.FromJsDate( Info.mTime );

    var lResponse := new TQTXFileInfoResponse(lRequest.Ticket);
    lResponse.Routing.TagValue := request.Routing.TagValue;
    lResponse.UserName := lUserName;
    lResponse.FileName := lFileName;
    lResponse.Assign(lData);

    try
      Socket.Send( lResponse.Serialize() );
    except
      on e: exception do
        WriteToLog(e.message);
    end;
  end);
end;

procedure TQTXTaskService.HandleDestroyDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lMessage := TQTXFileDestroyDeviceRequest(request);

  // This will also destroy any files + unregister the device in the
  // database table for the service -- do not mess with this!
  UnRegisterLocalDevice(nil, lMessage.Username, lMessage.DeviceName,
  procedure (TagValue: variant; LocalPath: string; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.Message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    var lResponse := TQTXFileDestroyDeviceResponse.Create(request.ticket);
    lResponse.UserName := lMessage.UserName;
    lResponse.DeviceName := lMessage.DeviceName;
    lResponse.Routing.TagValue := Request.Routing.TagValue;
    lResponse.Code := CNT_MESSAGE_CODE_OK;
    lResponse.Response := CNT_MESSAGE_TEXT_OK;

    try
      Socket.Send( lResponse.Serialize() );
    except
      on e: exception do
      begin
        WriteToLog(e.message);
      end;
    end;
  end);
end;

procedure TQTXTaskService.HandleCreateLocalDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lMessage := TQTXFileCreateLocalDeviceRequest(request);

  // Attempt to register.
  // NOTE: This will automatically create a matching folder
  //       under $cwd/userdevices/[calculated_name_of_device]

  RegisterLocalDevice(nil, lMessage.Username, lMessage.DeviceName,
  procedure (TagValue: variant; LocalPath: string; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.Message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    FindDeviceByName(nil, lMessage.Username, lMessage.DeviceName,
    procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
    begin
      if Error  nil then
      begin
        WriteToLog(Error.Message);
        SendError(Socket, Request, Error.Message);
        exit;
      end;

      var lResponse := TQTXFileCreateLocalDeviceResponse.Create(request.ticket);
      lResponse.UserName := lMessage.UserName;
      lResponse.Routing.TagValue := Request.Routing.TagValue;
      lResponse.Code := CNT_MESSAGE_CODE_OK;
      lResponse.Response := CNT_MESSAGE_TEXT_OK;
      if Device  nil then
        lResponse.assign(Device);

      try
        Socket.Send( lResponse.Serialize() );
      except
        on e: exception do
        begin
          WriteToLog(e.message);
        end;
      end;

    end);
  end);
end;

procedure TQTXTaskService.HandleGetDeviceByName(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lMessage := TQTXFileGetDeviceByNameRequest(request);

  FindDeviceByName(nil, lMessage.Username, lMessage.DeviceName,
  procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.Message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    var lResponse := TQTXFileGetDeviceByNameResponse.Create(request.ticket);
    lResponse.UserName := lMessage.UserName;
    lResponse.Code := CNT_MESSAGE_CODE_OK;
    lResponse.Response := CNT_MESSAGE_TEXT_OK;
    if Device  nil then
      lResponse.assign(Device);

    try
      Socket.Send( lResponse.Serialize() );
    except
      on e: exception do
      begin
        WriteToLog(e.message);
      end;
    end;
  end);

end;

procedure TQTXTaskService.HandleGetDevices(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
  var lMessage := TQTXFileGetDeviceListRequest(Request);
  GetDevicesForUser(nil, lMessage.Username,
  procedure (TagValue: variant; Devices: JDeviceList; Error: Exception)
  begin
    if Error  nil then
    begin
      WriteToLog(Error.Message);
      SendError(Socket, Request, Error.Message);
      exit;
    end;

    var lResponse := TQTXFileGetDeviceListResponse.Create(request.ticket);
    lResponse.UserName := lMessage.UserName;
    lResponse.Code := CNT_MESSAGE_CODE_OK;
    lResponse.Response := CNT_MESSAGE_TEXT_OK;
    if Devices  nil then
      lResponse.assign(Devices);

    try
      Socket.Send( lResponse.Serialize() );
    except
      on e: exception do
      begin
        WriteToLog(e.message);
      end;
    end;

  end);
end;

procedure TQTXTaskService.AfterServerStarted;
begin
  inherited;

  // Check prefs if zconfig should be applied
  if self.FPrefs.ReadBoolean("zconfig", "active", false) then
  begin
    // ZConfig should only run on the master instance.
    // We dont want to register our endpoint for each worker
    if NodeJSClusterAPI().isWorker then
      exit;

    writeln("Setting up Zero-Configuration layer");
    FZConfig.port := FPrefs.ReadInteger('zconfig', 'bindport', 2109);
    FZConfig.address := GetMachineIP();
    FZConfig.Start(nil, procedure (Sender: TObject; TagValue: variant; Error: Exception)
    begin
      if FPrefs.ReadBoolean("zconfig", "broadcast", true) then
        FZConfig.Socket.setBroadcast(true);

      // Build up the endpoint (URL) for our websocket server
      var lEndpoint := '';

      if FPrefs.ReadBoolean('networking', 'secure', false) then
        lEndpoint := 'wss://'
      else
        lEndpoint := 'ws://';

      lEndpoint += GetMachineIP();
      lEndpoint += ':' + Port.ToString();

      // Ping the ZConfig service on interval, until our service is registered
      // We keep track of the interval handle so we can stop calling on interval later
      FRegHandle := TQTXDispatch.SetInterval( procedure ()
      begin
        inc(FRegCount);

        // Only output once to avoid overkill in the log
        if FRegCount = 1 then
          WriteToLogF("ZConfig registration begins [%s]", [lEndpoint]);

        FZConfig.RegisterService(nil, CNT_ZCONFIG_SERVICE_NAME, SERVICE_ID_TASKMANAGER, lEndpoint,
        procedure (TagValue: variant; Error: Exception)
        begin
          if Error = nil then
          begin
            WriteToLog("Service registered");
            TQTXDispatch.ClearInterval(FRegHandle);
            FRegCount := 0;
            exit;
          end;
        end);
      end, 1000);

    end);
  end;
end;

procedure TQTXTaskService.BeforeServerStopped;
begin
  inherited;
end;

procedure TQTXTaskService.Dispatch(Socket: TNJWebSocketSocket; Message: TQTXBaseMessage);
begin
  var LInfo := MessageDispatch.GetMessageInfoForClass(Message);
  if LInfo  nil then
  begin
    try
      LInfo.MessageHandler(Socket, Message);
    except
      on e: exception do
      begin
        //Log error
        WriteToLog(e.message);
      end;
    end;
  end;
end;

end.


 

Quartex “Cloud Ripper” hardware

November 10, 2019 Leave a comment

For close to a year now I have been busy on a very exciting project, namely my own cloud system. While I have written about this project quite a bit these past months, mostly focusing on the software aspect, not much has been said about that hardware.

74238389_10156646805205906_1728576808808349696_o

Quartex “Cloud Ripper” running neatly on my home-office desk

So let’s have a look at Cloud Ripper, the official hardware setup for Quartex Media Desktop.

Tiny footprint, maximum power

Despite its complexity, the Quartex Media Desktop architecture is surprisingly lightweight. The services that makes up the baseline system (read: essential services) barely consume 40 megabytes of ram per instance (!). And while there is a lot of activity going on between these services -most of that activity is message-dispatching. Sending messages costs practically nothing in cpu and network terms. This will naturally change the moment you run your cloud as a public service, or setup the system in an office environment for a team. The more users, the more signals are shipped between the processes – but with the exception of reading and writing large files, messages are delivered practically instantaneous and hardly use CPU time.

CloudRipper

Quartex Media Desktop is based on a clustered micro-service architecture

One of the reasons I compile my code to JavaScript (Quartex Media Desktop is written from the ground up in Object Pascal, which is compiled to JavaScript) has to do with the speed and universality of node.js services. As you might know, Node.js is powered by the Google V8 runtime engine, which means the code is first converted to bytecodes, and further compiled into highly optimized machine-code [courtesy of llvm]. When coded right, such Javascript based services execute just as fast as those implemented in a native language. There simply are no perks to be gained from using a native language for this type of work. There are however plenty of perks from using Node.js as a service-host:

  • Node.js delivers the exact same behavior no matter what hardware or operating-system you are booting up from. In our case we use a minimal Linux setup with just enough infrastructure to run our services. But you can use any OS that supports Node.js. I actually have it installed on my Android based Smart-TV (!)
  • We can literally copy our services between different machines and operating systems without recompiling a line of code. So we don’t need to maintain several versions of the same software for different systems.
  • We can generate scripts “on the fly”, physically ship the code over the network, and execute it on any of the machines in our cluster. While possible to do with native code, it’s not very practical and would raise some major security concerns.
  • Node.js supports WebAssembly, you can use the Elements Compiler from RemObjects to write service modules that executes blazingly fast yet remain platform and chipset independent.

The Cloud-Ripper cube

The principal design goal when I started the project, was that it should be a distributed system. This means that instead of having one large-service that does everything (read: a typical “native” monolithic design), we instead operate with a microservice cluster design. Services that run on separate SBC’s (single board computers). The idea here is to spread the payload over multiple mico-computers that combined becomes more than the sum of their parts.

IMG_4644_Product_1024x1024@2x

Cloud Ripper – Based on the Pico 5H case and fitted with 5 x ODroid XU4 SBC’s

So instead of buying a single, dedicated x86 PC to host Quartex Media Desktop, you can instead buy cheap, off-the-shelves, easily available single-board computers and daisy chain them together. So instead of spending $800 (just to pin a number) on x86 hardware, you can pick up $400 worth of cheap ARM boards and get better network throughput and identical processing power (*). In fact, since Node.js is universal you can mix and match between x86, ARM, Mips and PPC as you see fit. Got an older PPC Mac-Mini collecting dust? Install Linux on it and get a few extra years out of these old gems.

(*) A single XU4 is hopelessly underpowered compared to an Intel i5 or i7 based PC. But in a cluster design there are more factors than just raw computational power. Each board has 8 CPU cores, bringing the total number of cores to 40. You also get 5 ARM Mali-T628 MP6 GPUs running at 533MHz. Only one of these will be used to render the HTML5 display, leaving 4 GPUs available for video processing, machine learning or compute tasks. Obviously these GPUs won’t hold a candle to even a mid-range graphics card, but the fact that we can use these chips for audio, video and computation tasks makes the system incredibly versatile.

Another design goal was to implement a UDP based Zero-Configuration mechanism. This means that the services will find and register with the core (read: master service) automatically, providing the machines are all connected to the same router or switch.

IMG_4650_Product_1024x1024@2x

Put together your own supercomputer for less than $500

The first “official” hardware setup is a cluster based on 5 cheap ARM boards; namely the ODroid XU4. The entire setup fits inside a Pico Cube, which is a special case designed to house this particular model of single board computers. Pico offers several different designs, ranging from 3 boards to a 20 board super-cluster. You are not limited ODroid XU4 boards if you prefer something else. I picked the XU4 boards because they represent the lowest possible specs you can run the Quartex Media Desktop on. While the services themselves require very little, the master board (the board that runs the QTXCore.js service) is also in charge of rendering the HTML5 display. And having tested a plethora of boards, the ODroid XU4 was the only model that could render the desktop properly (at that low a price range).

Note: If you are thinking about using a Raspberry PI 3B (or older) as the master SBC, you can pretty much forget it. The media desktop is a piece of very complex HTML5, and anything below an ODroid XU4 will only give you a terrible experience (!). You can use smaller boards as slaves, meaning that they can host one of the services, but the master should preferably be an ODroid XU4 or better. The ODroid N2 [with 4Gb Ram] is a much better candidate than a Raspberry PI v4. A Jetson Nano is an even better option due to its extremely powerful GPU.

Booting into the desktop

One of the things that confuse people when they read about the desktop project, is how it’s possible to boot into the desktop itself and use Quartex Media Desktop as a ChromeOS alternative?

How can a “cloud platform” be used as a desktop alternative? Don’t you need access to the internet at all times? If it’s a server based system, how then can we boot into it? Don’t we need a second PC with a browser to show the desktop?

73475069_10156646805615906_2668445017588105216_o

Accessing the desktop like a “web-page” from a normal Linux setup

To make a long story short: the “master” in our cluster architecture (read: the single-board computer defined as the boss) is setup to boot into a Chrome browser display under “kiosk mode”. When you start Chrome in kiosk mode, this removes all traces of the ordinary browser experience. There will be no toolbars, no URL field, no keyboard shortcuts, no right-click popup menus etc. It simply starts in full-screen and whatever HTML5 you load, has complete control over the display.

What I have done, is to to setup a minimal Linux boot sequence. It contains just enough Linux to run Chrome. So it has all the drivers etc. for the device, but instead of starting the ordinary Linux Desktop (X or Wayland) -we instead start Chrome in kiosk mode.

74602781_10156646805300906_6294526665393438720_o

Booting into the same desktop through Chrome in Kiosk Mode. In this mode, no Linux desktop is required. The Linux boot sequence is altered to jump straight into Chrome

Chrome is started to load from 127.0.0.1 (this is a special address that always means “this machine”), which is where our QTXCore.js service resides that has it’s own HTTP/S and Websocket servers. The client (HTML5 part) is loaded in under a second from the core — and the experience is more or less identical to starting your ChromeBook or NAS box. Most modern NAS (network active storage) devices are much more than a file-server today. NAS boxes like those from Asustor Inc have HDMI out, ships with a remote control, and are designed to act as a media center. So you connect the NAS directly to your TV, and can watch movies and listen to music without any manual conversion etc.

In short, you can setup Quartex Media Desktop to do the exact same thing as ChromeOS does, booting straight into the web based desktop environment. The same desktop environment that is available over the network. So you are not limited to visiting your Cloud-Ripper machine via a browser from another computer; nor are you limited to just  using a dedicated machine. You can setup the system as you see fit.

Why should I assemble a Cloud-Ripper?

Getting a Cloud-Ripper is not forced on anyone. You can put together whatever spare hardware you have (or just run it locally under Windows). Since the services are extremely lightweight, any x86 PC will do. If you invest in a ODroid N2 board ($80 range) then you can install all the services on that if you like. So if you have no interest in clustering or building your own supercomputer, then any PC, Laptop or IOT single-board computer(s) will do. Provided it yields more or equal power as the XU4 (!)

What you will experience with a dedicated cluster, regardless of putting the boards in a nice cube, is that you get excellent performance for very little money. It is quite amazing what $200 can buy you in 2019. And when you daisy chain 5 ODroid XU4 boards together on a switch, those 5 cheap boards will deliver the same serving power as an x86 setup costing twice as much.

Jetson-Nano_3QTR-Front_Left_trimmed

The NVidia Jetson Nano SBC, one of the fastest boards available at under $100

Pico is offering 3 different packages. The most expensive option is the pre-assembled cube. This is for some reason priced at $750 which is completely absurd. If you can operate a screwdriver, then you can assemble the cube yourself in less than an hour. So the starter-kit case which costs $259 is more than enough.

Next, you can buy the XU4 boards directly from Hardkernel for $40 a piece, which will set you back $200. If you order the Pico 5H case as a kit, that brings the sub-total up to $459. But that price-tag includes everything you need except sd-cards. So the kit contains power-supply, the electrical wiring, a fast gigabit ethernet switch [built-into the cube], active cooling, network cables and power cables. You don’t need more than 8Gb sd-cards, which costs practically nothing these days.

Note: The Quartex Media Desktop “file-service” should have a dedicated disk. I bought a 256Gb SSD disk with a USB 3.0 interface, but you can just use a vanilla USB stick to store user-account data + user files.

As a bonus, such a setup is easy to recycle should you want to do something else later. Perhaps you want to learn more about Kubernetes? What about a docker-swarm? A freepascal build-server perhaps? Why not install FreeNas, Plex, and a good backup solution? You can set this up as you can afford. If 5 x ODroid XU4 is too much, then get 3 of them instead + the Pico 3H case.

So should Quartex Media Desktop not be for you, or you want to do something else entirely — then having 5 ODroid XU4 boards around the house is not a bad thing.

Oh and if you want some serious firepower, then order the Pico 5H kit for the NVidia Jetson Nano boards. Graphically those boards are beyond any other SoC on the market (in it’s price range). But as a consequence the Jetson Nano starts at $99. So for a full kit you will end up with $500 for the boards alone. But man those are the proverbial Ferrari of IOT.

Quartex Media Desktop, new compiler and general progress

September 11, 2019 3 comments

It’s been a few weeks since my last update on the project. The reason I dont blog that often about Quartex Media Desktop (QTXMD), is because the official user-group has grown to 2000+ members. So it’s easier for me to post developer updates directly to the audience rather than writing articles about it.

desktop_01

Quartex Media Desktop ~ a complete environment that runs on every device

If you haven’t bothered digging into the project, let me try to sum it up for you quickly.

Quick recap on Quartex Media Desktop

To understand what makes this project special, first consider the relationship between Microsoft Windows and a desktop program. The operating system, be it Windows, Linux or OSX – provides an infrastructure that makes complex applications possible. The operating-system offers functions and services that programs can rely on.

The most obvious being:

  • A filesystem and the ability to save and load data
  • A windowing toolkit so programs can be displayed and have a UI
  • A message system so programs can communicate with the OS
  • A service stack that takes care of background tasks
  • Authorization and identity management (security)

I have just described what the Quartex Media Desktop is all about. The goal is simple:

to provide for JavaScript what Windows and OS X provides for ordinary programs.

Just stop and think about this. Every “web application” you have ever seen, have all lacked these fundamental features. Sure you have libraries that gives you a windowing environment for Javascript, like Embarcadero Sencha; but im talking about something a bit more elaborate. Creating windows and buttons is easy, but what about ownership? A runtime environment has to keep track of the resources a program allocates, and make sure that security applies at every step.

Target audience and purpose

Take a second and think about how many services you use that have a web interface. In your house you probably have a router, and all routers can be administered via the browser. Sadly, most routers operate with a crude design and that leaves much to be desired.

router

Router interfaces for web are typically very limited and plain looking. Imagine what NetGear could do with Quartex Media Desktop instead

If you like to watch movies you probably have a Plex or Kodi system running somewhere in your house; perhaps you access that directly via your TV – or via a modern media system like Playstation 4 or XBox one. Both Plex and Kodi have web-based interfaces.

Netflix is now omnipresent and have practically become an institution in it’s own right. Netflix is often installed as an app – but the app is just a thin wrapper around a web-interface. That way they dont have to code apps for every possible device and OS out there.

If you commute via train in Scandinavia, chances are you buy tickets on a kiosk booth. Most of these booths run embedded software and the interface is again web based. That way they can update the whole interface without manually installing new software on each device.

plex-desktop-movies-1024x659

Plex is a much loved system. It is based on a mix of web and native technologies

These are just examples of web based interfaces you might know and use; devices that leverage web technology. As a developer, wouldn’t it be cool if there was a system that could be forked, adapted and provide advanced functionality out of the box?

Just imagine a cheap Jensen router with a Quartex Media Desktop interface! It could provide a proper UI interface with applications that run in a windowing environment. They could disable ordinary desktop functionality and run their single application in kiosk mode. Taking full advantage of the underlying functionality without loss of security.

And the same is true for you. If you have a great idea for a web based application, you can fork the system, adjust it to suit your needs – and deploy a cutting edge cloud system in days rather than months!

New compiler?

Up until recently I used Smart Mobile Studio. But since I have left that company, the matter became somewhat pressing. I mean, QTXMD is an open-source system and cant really rely on third-party intellectual property. Eventually I fired up Delphi, forked the latest  DWScript, and used that to roll a new command-line compiler.

desktop_02

Web technology has reached a level of performance that rivals native applications. You can pretty much retire Photoshop in favour of web based applications these days

But with a new compiler I also need a new RTL. Thankfully I have been coding away on the new RTL for over a year, but there is still a lot of work to do. I essentially have to implement the same functionality from scratch.

There will be more info on the new compiler / codegen when its production ready.

Progress

If I was to list all the work I have done since my last post, this article would be a small book. But to sum up the good stuff:

  • Authentication has been moved into it’s own service
  • The core (the main server) now delegates login messages to said service
  • We no longer rely on the Smart Pascal filesystem drivers, but use the raw node.js functions instead  (faster)
  • The desktop now use the Smart Theme engine. This means that we can style the desktop to whatever we like. The OS4 theme that was hardcoded will be moved into its own proper theme-file. This means the user can select between OS4, iOS, Android and Ubuntu styling. Creating your own theme-files is also possible. The Smart theme-engine will be replaced by a more elaborate system in QTX later
  • Ragnarok (the message api) messages now supports routing. If a routing structure is provided,  the core will relay the message to the process in question (providing security allows said routing for the user)
  • The desktop now checks for .info files when listing a directory. If a file is accompanied by an .info file, the icon is extracted and shown for that file
  • Most of the service layer now relies on the QTX RTL files. We still have some dependencies on the Smart Pascal RTL, but we are making good progress on QTX. Eventually  the whole system will have no dependencies outside QTX – and can thus be compiled without any financial obligations.
  • QTX has it’s own node.js classes, including server and client base-classes
  • Http(s) client and server classes are added to QTX
  • Websocket and WebSocket-Secure are added to QTX
  • TQTXHybridServer unifies http and websocket. Meaning that this server type can handle both orinary http requests – but also websocket connections on the same network socket. This is highly efficient for websocket based services
  • UDP classes for node.js are implemented, both client and server
  • Zero-Config classes are now added. This is used by the core for service discovery. Meaning that the child services hosted on another machine will automatically locate the core without knowing the IP. This is very important for machine clustering (optional, you can define a clear IP in the core preferences file)
  • Fixed a bug where the scrollbars would corrupt widget states
  • Added API functions for setting the scrollbars from hosted applications (so applications can tell the desktop that it needs scrollbar, and set the values)
  • .. and much, much more

I will keep you all posted about the progress — the core (the fundamental system) is set for release in december – so time is of the essence! Im allocating more or less all my free time to this, and it will be ready to rock around xmas.

When the core is out, I can focus solely on the applications. Everything from Notepad to Calculator needs to be there, and more importantly — the developer tools. The CloudForge IDE for developers is set for 2020. With that in place you can write applications for iOS, Android, Windows, OS X and Linux directly from Quartex Media Desktop. Nothing to install, you just need a modern browser and a QTX account.

The system is brilliant for small teams and companies. They can setup their own instance, communicate directly via the server (text chat and video chat is scheduled) and work on their products in concert.

New job, new office, new adventures

May 12, 2019 5 comments

It’s been roughly 4 weeks since I posted a status report on Amibian.js. I normally keep people up-to-date on facebook (the “Amiga Disrupt” and also “Delphi Developer” groups). It’s been a very hectic month so I fully understand that people are asking. So let’s look at where the project is at and where we are on the time-line.

For those that might not know, I decided to leave Embarcadero a couple of months ago. I will be working out may before I move on. I wanted to write about that myself in a clean fashion, but sadly the news broke on Facebook prematurely.

Long story short, I have been very fortunate to work at Embarcadero. I am not leaving because there is anything wrong or something like that. I was hired as SC for the EMEA regions, which basically made me the support and presenter for most of europe, parts of asia and the middle east. It’s been a great adventure, but ultimately I had to admit that my passion is coding and community work. Sales is a very important part of any company, but it’s not really my cup of tea; my passion has always been research and development.

So, come first of June and I start in a new position at RemObjects. A company that has deep roots with Delphi and C++ builder users – and a company that continues to produce a wealth of high-quality, high-performance frameworks for Delphi and C++ builder. RemObjects also has a strong focus on modern languages, and have a strong portfolio of new and exciting compilers and languages to offer. The Oxygene compiler should be no stranger to Delphi developers, a powerful object-pascal dialect that can target a variety of platforms and chipsets.

Since compiler technology and run-time systems has been my main focus for well over a decade now, I feel RemObjects is a better match.

Quartex Components

Quartex Components has been an officially registered Norwegian company for a while now, so perhaps not news. What is news is that it’s now directly connected with the development of the Quartex Media Desktop (codename “Amibian.js”). While Amibian.js is an open source endeavour, there will be both free and commercial products running on top of that platform. I have written at length about Cloud Forge in the past, so I wont re-hash that again. But 2020 will see a paradigm shift in how teams and companies approach software development.

quartex

Company logo professionally milled and on its way to my new office

I will also, once there is more time, continue to sell and support software license components.

Quartex Media Desktop

The “Amibian.js” project is moving along nicely. The deadline is Q4 2019, but im hoping to wrap up the core functionality before that. So we are on track and kicking ass 🙂

amibian_01

More and more elaborate functionality is being implemented for the desktop

Here is an overview of work done this month:

  • TSystemService application type has been created (node.js)
    • TApplication now holds IPC functions (inter process communication)
    • Running child processes + sending messages is now simplicity itself
    • Database drivers are 90% done. Delete() and DeleteTable() functionality needs to be implemented in a uniform way
  • Authentication is now a separate service
    • Service database layer is finished (using SQLite3 driver by default)
    • Authentication protocol has been designed
    • Server protocol and JSON message envelopes are done
    • Presently working on the client interface
  • LDEF bytecode assembler has been improved
    • Faster symbolic lookup
    • Smarter register recognition
    • Early support for stack-frames
    • Fixed bug in parser (comma-list parse)
  • QTX framework has seen a lot of work
    • Large parts of the RTL sub-strata has been implemented
    • UTF16 codec implemented
    • QTX versions of common controls:
      • TQTXButton
      • TQTXLabel
      • TQTXToolbar
        • TQTXToolButton
        • TQTXToolSeparator
        • TQTXToolElement
      • TQTXPanel
      • TQTXCheckBox
      • .. and much, much more
  • Desktop changes
    • Link Maker functionality has been added
    • Handshake process between desktop and child app now runs on a separate timer, ensuring better conformity and a more robust initialization
    • The Quartex Editor control has been optimized
      • All redraw calls are now synchronized
      • Canvas is created on demand, avoids flicker during initial redraw
      • Support for DEL key + behavior
      • Gutter is now rendered to an offscreen bitmap and blitted into the control’s canvas. The gutter is only fully rendered when cursor forces the view to change

I will continue to keep everyone up to date about the project. As you can understand, its a bit hectic right now so please be patient – it is turning into an EPIC environment!

Quartex: Mali GPU glitches

February 20, 2019 Leave a comment

EDIT: I did further testing after this article was written, and believe the source of this to be about heat. Even with extra fans, running games like Tyrian (asm.js) that are extremely demanding, plus resizing a graphics intensive windows constantly, the temperature reached 71 degrees C very quickly. And this was with two cabinet fans helping the built-in fan to cool the device. It is thus not unthinkable that when running solo (no extra fans) that the kernel shut the device down to not cook the chipset. Which also explains why the device wont boot properly afterwards (the device is still hot).

Glitches

Something really strange is happening on Chrome and Firefox for ARM. JavaScript is not supposed to be able to take down a system, and in this case it’s neither an attempt as such either — yet for some reason I have managed to take down the ODroid XU4 with both Chrome and Firefox lately.

ODroid XU4

I guess I should lead with that I’m not able to replicate this on x86. One of the things I really love about the ODroid XU4 is that it’s affordable, powerful and probably the only SBC I have used that runs stable on the mali GPU. As you probably know I tested at least 10 different SBC’s back in 2018, and whenever there was a mali GPU involved, the product was either haunted by instabilities or lacked drivers all together.

amibian

Since the codebase for Chrome (and I presume Firefox) is ultimately the same between platforms, it leaves a question-mark about the ODroid. It is by far the most stable SBC I have tested so far (except for the PI, which is sadly underpowered for this task), but stable doesn’t mean flawless. And to be honest, Amibian.js is pushing web tech to the very limits.

Not Mali again

The reason I suspect the mali to be the culprit behind all this, is because the “bug” if we can call it that, happens exclusively during resize. So if there is a lot going on inside a desktop-window, you can sometimes provoke the ODroid to cold-crash and reboot. You actually have to power the board down and switch it back on for it to boot properly.

50431451_10155954273110906_8776790185049325568_n

Cloudripper ~ 5x ODroid XU4 [40 cores] in a PICO 5h cube

The resize and moving of windows uses CSS transformation, which in modern browsers makes use of the GPU. Chrome talks directly with OpenGL (or glES), so the operations are proxied through that. And again, since OpenGL is pretty rock solid elsewhere, we are only left with one common denominator: the mali GPU.

The challenge is that there is no way to debug or catch this error, because when it occurs the whole system literally goes down. There is no exception thrown, nor is the browser process terminated (not even a log entry, so it’s a clean-cut) — the system reboots on the spot. Since it fails on reboot when opening X (setting a screen-mode) I again point the finger at the GPU. Somehow a flag or lock survives the cold-reboot and that’s why you have to manually switch it off and on again.

This is the exact problem that made the NanoPI Fire useless. It only shipped with Android embedded drivers. The X drivers could hardly open a display without crashing. Such a waste of a good cpu.

x86 as head

ODroid is perfect for a low-cost Amibian.js experience, but I was unsure if it would handle the payload. Interestingly it handles it just fine and even with a high-speed action game running + background tasks we are not using 50% of the CPU even.

Ram is holding up too, with memory consumption while running Tyrian + having a few graphics viewers open, is at a reasonable 700 mb (of 2 gigabyte in total).

51398321_10155998598505906_8984850199142727680_o

Tyrian jogs along at 45 fps ~ that is not bad for a $45 SBC

Right now this strange error is rare, but if it continues or grows into a problem (chrome is hardly useable at all, only firefox) then I have no option than to replace the master sbc in the cluster with something else. The x86 UP board is more than capable, but it would be a shame to break the price range because of that (excuse my language) crap mali GPU. I honestly don’t understand why board makers insist on using a mali. Every board that has a mali is haunted by problems and get poor reviews.

It will be exciting to check out the dragonboard, although I fear 1Gb memory will not be enough for smooth operation. Not without a sata interface and a good swap-file.

Android and Delphi

One alternative is to switch to Android and use Delphi to code a custom Chromium Embedded webview. I am hoping to avoid the overhead of Android, but Delphi would definitively be a bonus with Android embedded (“Android of things”).

We will see.

Smart Pascal + WebOS = true

April 15, 2017 Leave a comment

LG-WebOSIf you own a television (and who doesn’t) chances are you own an LG model. The LG brand has been on the market since before recorded history (or so it feels) and have been a major player for decades.

What is little known though, is that LG also owns and finance a unique operating system. This is not uncommon these days, most NAS and cloud vendors have their own system (there are roughly 20 cloud based systems on the market that I know of). The only way to make a Smart TV (sigh) is to add a computer to it. And if you buy a television today chances are it contains a small embedded board running some custom-made operating system designed to do just that.

Every vendor has their own system, and those that don’t usually end up forking Android and adapt that to their needs.

Luna WebOS

LG’s system is called WebOS. This may sound like yet another html eye candy front end, but hear me out because this OS is not what it seems.

VirtualBox_Developers LuneOS emulator appliance 20151006131924-stable-038-253_15_04_2017_16_01_43

Except for some font issues (see disk label) Smart loaded fine under WebOS

Remember Palm OS? If you are between 35 and 45 you should remember that before Apple utterly demolished the mobile-phone market with their iPhone, one of the most popular brands was Palm. Their core product being “Palm Pilot”, a kind of digital filo-fax (personal organizer) and mobile phone rolled into one.

WebOS is based on what used to be called Palm-OS, but it has been completely revamped, given a sexy new user interface (looks a lot like android to be honest!) and brought into the present age. And best of all: its 100% free! It runs on a plethora of systems, from x86 to Raspberry PI to Mips. It is used in televisions, terminals and set-top-boxes around the world – and is quite popular amongst engineers.

Check it out their new portal here; there is also plenty of links to pre-built images if you look around there: https://pivotce.com/

JavaScript application stack

One of the coolest features in my view, is that their applications are primarily made by JavaScript. The OS itself is native of course, but they have discovered that JavaScript and HTML5 is an excellent way to build applications. Applications that is easy to control, sandboxed and safe yet incredibly powerful and capable.

VirtualBox_Developers LuneOS emulator appliance 20151006131924-stable-038-253_15_04_2017_16_00_46

Smart Desktop booting Quake 3 in Luna OS

Well, today I sent them an Email requesting their SDK. I know they use Enyo.js as their primary and suggested framework – but that is no problem for Smart Mobile Studio. A better question is: can Luna handle our codebase?

When I loaded up Quake 3 in pure Asm.JS the TV crashed with a spectacular access violation. So they are probably not used to the level of hardcore coding Smart developers represent. But yeah, Quake III is about as advanced as it gets – and it pushes any browser to the outer limits of what is possible.

Once I get the SDK, docs and a few examples – I will begin adding support for it as quickly as possible. Which means you get a new project type especially for the platform + API units (typically stored under $RTL\Apis folder).

Why is this cool?

Because with support for Luna / WebOS, you as a Delphi or Smart developer can offer your services to companies that use WebOS or Luna (the open source version) in their devices. The list of companies is substantial – and they are well established companies. And as you have no doubt noticed, hardware engineers doesn’t always make the best software engineers. So this opens up plenty of opportunities for a good object pascal / Smart developer.

17948597_10154372538120906_1912877833_o

Luna has an Android like quality over it – except its more smooth to use

Remember – other developers will use vanilla JavaScript. You have the onslaught of the VJL and the might of our libraries at your disposal. You can produce in days what others use weeks on achieving.

These are exciting days! I’ll keep you posted on our progress!

Smart Pascal, the next generation

April 15, 2017 1 comment

I want to take the time to talk a bit about the future, because like all production companies we are all working towards lesser and greater goals. If you don’t have a goal then you are in trouble; Thankfully our goals have been very clear from the beginning. Although I must admit that our way there has been.. “colorful” at times.

When we started back in 2010 we didn’t really know what would become of our plans. We only knew that this was important; there was a sense of urgency and “we have to build this” in the air; I think everyone involved felt that this was the case, without any rational explanation as to why. Like all products of passion it can consume you in a way – and you work day and night on turning an idea into something real. From the intangible to the tangible.

transitions_callback

It seems like yesterday, but it was 5 years ago!

By the end of 2011 / early 2012, Eric and myself had pretty much proven that this could be done. At the time there were more than enough nay-sayers and I think both of us got flamed quite often for daring to think different. People would scoff at me and say I was insane to even contemplate that object pascal could ever be implemented for something as insignificant and mediocre as JavaScript. This was my first meeting with a sub-culture of the Delphi and C++ community, a constellation I have gone head-to-head with on many occasions. But they have never managed to shake my resolve as much as inch.

 

 

When we released version 1.0 in 2012 some ideas about what could be possible started to form. Jørn defined plans for a system we later dubbed “Smart net”. In essence it would be something you logged onto from the IDE – allowing you to store your projects in the cloud, compile in the cloud (connected with Adobe build services) and essentially move parts of your eco-system to the cloud. Keep in mind this was when people still associated cloud with “storage”; they had not yet seen things like Uber or Netflix or played Quake 3 at 160 frames per second, courtesy of asm.js in their browser.

The second part would be a website where you could do the same, including a live editor, access to the compiler and also the ability to buy and sell components, solutions and products. But for that we needed a desktop environment (which is where the Quartex Media Desktop came in later).

cool

The first version of the Media Desktop, small but powerful. Here running in touch-screen mode with classical mobile device layout (full screen forms).

Well, we have hit many bumps along the road since then. But I must be honest and say, some of our detours have also been the most valuable. Had it not been for the often absurd (to the person looking in) research and demo escapades I did, the RTL wouldn’t be half as powerful as it is today. It would deliver the basics, perhaps piggyback on Ext.js or some lame, run of the mill framework – and that would be that. Boring, flat and limited.

What we really wanted to deliver was a platform. Not just a website, but a rich environment for creating, delivering and enjoying web and cloud based applications. And without much fanfare – that is ultimately what the Smart Desktop and it’s sexy node.js back-end is all about is all about.

We have many project types in the pipeline, but the Smart Desktop type is by far the most interesting and powerful. And its 100% under your control. You can create both the desktop itself as a project – and also applications that should run on that desktop as separate projects.

This is perfectly suited for NAS design (network active storage devices can usually be accessed through a web portal on the device), embedded boards, intranets and even intranets for that matter.

You get to enjoy all the perks of a multi-user desktop, one capable of both remote desktop access, telnet access, sharing files and media, playing music and video (we even compiled the mp4 codec from C to JavaScript so you can play mp4 movies without the need for a server backend).

The Smart Desktop

The Smart Desktop project is not just for fun and games. We have big plans for it. And once its solid and complete (we are closing in on 46% done), my next side project will not be more emulators or demos – it will be to move our compiler(s) to Amazon, and write the IDE itself from scratch in Smart Pascal.

smart desktop

The Smart Desktop – A full desktop in the true sense of the word

And yeah, we have plans for EmScripten as well – which takes C/C++ and compiles it into asm.js. It will take a herculean effort to merge our RTL with their sandboxed infrastructure – but the benefits are too great to ignore.

As a bonus you get to run native 68k applications (read: Amiga applications) via emulation. While I realize this will be mostly interesting for people that grew up with that machine – it is still a testament to the power of Smart and how much you can do if you really put your mind to it.

Naturally, the native IDE wont vanish. We have a few new directions we are investigating here – but native will absolutely not go anywhere. But cloud, the desktop system we are creating, is starting to become what we set out to make five years ago (has it been half a decade already? Tempus fugit!). As you all know it was initially designed as an example of how you could write full-screen applications for Raspberry PI and similar embedded devices. But now its a full platform in its own right – with a Linux core and node.js heart, there really is very little you cannot do here.

scsc

The Smart Pascal compiler is one of our tools that is ready for cloud-i-fication

Being able to login to the Smart company servers, fire up the IDE and just code – no matter if you are: be it Spain, Italy, Egypt, China or good old USA — is a pretty awesome thing!

Clicking compile and the server does the grunt work and you can test your apps live in a virtual window; switch between device layouts and targets — then hit “publish” and it goes to Cordova (or Delphi) and voila – you get a message back when binaries for 9 mobile devices is ready for deployment. One click to publish your applications on Appstore, Google play and Microsoft marketplace.

Object pascal works

People may have brushed off object pascal (and from experience those people have a very narrow view of what object pascal is all about), but when they see what Smart delivers, which in itself is written in Delphi, powered by Delphi and should be in every Delphi developer’s toolbox — i think it should draw attention to both Delphi as a product, object pascal as a language – and smart as a solution.

With Smart it doesn’t matter what computer you use. You can sit at home with the new A1222 PPC Amiga, or a kick-ass Intel i7 beast that chew virtual machines for breakfast. If your computer can handle a modern website, then you can learn object pascal and work directly in the cloud.

desktop_embedded

The Smart Desktop running on cheap embedded hardware. The results are fantastic and the financial savings of using Smart Pascal on the kiosk client is $400 per unit in this case

Heck you can work off a $60 ODroid XU4, it has more than enough horsepower to drive the latest chrome or Firefox engines. All the compilation takes place on the server anyways. And if you want a Delphi vessel rather than phonegap (so that it’s a Delphi application that opens up a web-view in full-screen and expose features to your smart code) then you will be happy to know that this is being investigated.

More targets

There are a lot of systems out there in the world, some of which did not exist just a couple of years ago. FriendOS is a cloud based operating system we really want to support, so we are eager to get cracking on their SDK when that comes out. Being able to target FriendOS from Smart is valuable, because some of the stuff you can do in SMS with just a bit of code – would take weeks to hand write in JavaScript. So you get a productive edge unlike anything else – which is good to have when a new market opens.

As far as Delphi is concerned there are smaller systems that Embarcadero may not be interested in, for example the many embedded systems that have come out lately. If Embarcadero tried to target them all – it would be a never-ending cat and mouse game. It seems like almost every month there is a new board on the market. So I fully understand why Embarcadero sticks to the most established vendors.

ov-4f-img

Smart technology allows you to cover all your bases regardless of device

But for you, the programmer, these smaller boards can repsent thousands of dollars worth of saving. Perhaps you are building a kiosk system and need to have a good-looking user interface that is not carved in stone, touch capabilities, low-latency full-duplex communication with a server; not much you can do about that if Delphi doesnt target it. And Delphi is a work horse so it demands a lot more cpu than a low-budget ARM SoC can deliver. But web-tech can thrive in these low-end environments. So again we see that Smart can compliment and be a valuable addition to Delphi. It helps you as a Delphi developer to act on opportunities that would otherwise pass you by.

So in the above scenario you can double down. You can use Smart for the user-interface on a low power, low-cost SoC (system on a chip) kiosk — and Delphi on the server.

It all depends on what you are interfacing with. If you have a full Delphi backend (which I presume you have) then writing the interface server in Delphi obviously makes more sense.

If you don’t have any back-end then, depending on your needs or future plans, it could be wise to investigate if node.js is right for you. If it’s not – go with what you know. You can make use of Smart’s capabilities either way to deliver cost-effective, good-looking device front-ends of mobile apps. Its valuable tool in your Delphi toolbox.

Better infrastructure and rooting

So far our support for various systems has been in the form of APIs or “wrapper units”. This is good if you are a low-level coder like myself, but if you are coming directly from Delphi without any background in web technology – you wouldn’t even know where to start.

So starting with the next IDE update each platform we support will not just have low-level wrapper units, but project types and units written and adapted by human beings. This means extra work for us – but that is the way it has to be.

As of writing the following projects can be created:

  • HTML5 mobile applications
  • HTML5 mobile console applications
  • Node.js console applications
  • node.js server applications
  • node.js service applications (requires PM2)
  • Web worker project (deprecated, web-workers can now be created anywhere)

We also have support for the following operating systems:

  • Chrome OS
  • Mozilla OS
  • Samsung Tizen OS

The following API’s have shipped with Smart since version 1.2:

  • Khronos browser extensions
  • Firefox spesific API
  • NodeWebkit
  • Phonegap
    • Phonegap provides native access to roughly 9 operating systems. It is however cumbersome to work with and beta-test if you are unfamiliar with the “tools of the trade” in the JavaScript world.
  • Whatwg
  • WAC Apis

Future goals

The first thing we need to do is to update and re-generate ALL header files (or pascal units that interface with the JavaScript libraries) and make what we already have polished, available, documented and ready for enterprise level use.

kiosk-systems

Why pay $400 to power your kiosk when $99 and Smart can do a better job?

Secondly, project types must be established where they make sense. Not all frameworks are suitable for full project isolation, but act more like utility libraries (like jQuery or similar training-wheels). And as much as possible of the RTL made platform independent and organized following our namespace scheme.

But there are also other operating systems we want to support:

  • Norwegian made Friend OS, which is a business oriented cloud desktop
  • Node.js OS is very exciting
  • LG WebOS, and their Enyo application framework
  • Asustor DLM web operating system is also a highly attractive system to support
  • OpenNAS has a very powerful JavaScript application framework
  • Segate Nas OS 4 likewise use JavaScript for visual, universal applications
  • Microsoft Universal Platform allows you to create truly portable, native speed JavaScript applications
  • QNap QTS web operating system [now at version 4.2]

All of these are separate from our own NAS and embedded device system: Smart Desktop, which uses node.js as a backend and will run on anything as long as node and a modern browser is present.

Final words

I hope you guys have enjoyed my little trip down memory lane, and also the plans we have for the future. Personally I am super excited about moving the IDE to the cloud and making Smart available 24/7 globally – so that everyone can use it to design, implement and build software for the future right now.

Smart Pascal Builder (or whatever nickname we give it) is probably the first of its kind in the world. There are a ton of “write code on the web” pages out there, but so far there is not a single hard-core development studio like I have in mind here.

So hold on, because the future is just around the corner 😉

Smart desktop: Amibian.js past, future and present

April 1, 2017 2 comments

Had someone told me 20 years ago that I would enter my 40’s enjoying JavaScript, my younger self would probably have beaten that someone over the head with a book on hardcore demo coding or something. So yeah, things have changed so much and we are right now in the middle of a paradigm shift that is taking us all to the next level – regardless if we agree or not.

Ask a person what he thinks about “cloud” and you rarely get an answer that resembles what cloud really is. Most will say “its a fancy way of dealing with storage”. Others will say its all about web-services – and others might say it’s about JavaScript.

memyselfandi

Old coders never die, we just get better

They are all right and wrong at the same time. Cloud is first of all the total abstraction of all parts that makes up a networked computer system. Storage, processing capacity, memory, operating system, services, programming language and even instruction set.

Where a programmer today works with libraries and classes to create programs that run on a desktop — dragging visual controls like edit-boxes and buttons into place using a form or window designer; a cloud developer builds whole infrastructures. He drags and drops whole servers into place, connects external and internal services.

Storage? Ok I need Dropbox, amazon, Google drive, Microsoft one disk, local disk – and he just drags that onto a module. Done. All of these services now have a common API that allows them to talk with each other. They become like dll files or classes, all built using the same mold – but with wildly different internals. It doesn’t matter as long as they deliver the functionality according to standard.

Processing power? Setup an Azure or Amazon account and if you got the cash, you can compute enough to pre-cacalculate the human brain if you like. It has all been turned into services — that you can link together like lego pieces.

Forget everything you think you know about web; that is but the visual rendering engine over the proverbial death-star of technology hidden beneath. People have only seen the tip of the ice berg.

Operating systems have been reduced to a preference. There is no longer any reason to pick Windows over Linux on the server. Microsoft knew years ago that this day would come. Back in the late 90s I remember reading an interview with Steve Balmer doing one of his black-ops meetings with Norwegian tech holders in Oslo; and he outlined software as a service when people were on 14.4 modems. He also mentioned that “we need a language that is universal” to make this a reality. You can guess why .net was stolen from Borland, not to mention their failed hostile takover of Java (or J#) which Anders Hejlsberg was hired to spear-head.

Amibian.js

Amibian.js is my, Gunnar and Thomas‘s effort to ensure that the Amiga is made portable and can be enjoyed in this new paradigm. It is not made to compete with anyone (as was suggested earlier), but rather to make sure Amiga gets some life into her again – and that people of this generation and the kids growing up now can get to enjoy the same exciting environment that we had.

Amibian666

From Scandinavia with love

The world is going JavaScript. Hardware now speaks JavaScript (!), your TV now speaks JavaScript – heck your digital watch probably runs JavaScript. And just to add insult to injury – asm.js now compiles JS code side-by-side with C/C++ in terms of speed. I think the browser has seen more man years of development time than any other piece of software out there – with the exception of GCC / Gnu Linux perhaps.

Amibian is also an example of a what you can do with Smart Pascal, which is a programming environment that compiles object pascal to JavaScript. One we (The Smart Company AS) made especially for this new paradigm. I knew this was coming years ago – and have been waiting for it. But that’s another story all together.

Future

Well, naturally the desktop system is written from scratch so it needs to be completed. We are at roughly 40% right now. But there is also work to be done on UAE.js (a mild fork of sae, scriptable Amiga emulator) in terms of speed and IO. We want the emulated Amiga side to interact with the modern web desktop – to be able to load and save files to whatever backend you are using.

 

Amibianstuff

For those about to rock; We salute you!

Well, it’s not that easy. UAE is like an organism, and introducing new organs into an existing creature is not easily done. UAE.js (or SAE) has omitted a huge chunk of the original code – which also includes the modified boot-code that adds support for external or “virtual” UAE drives (thanks to Frode Solheim of Fs-UAE for explaining the details of the parts here).

But, there are hacker ways. The dark side is a pathway to many abilities, some deemed unnatural. So if all else fails, i will kidnap the hardfile API and delegate the IO signals to the virtual filesystem on the server — in short, UAE.JS will think it’s booting from a hardfile in memory – when in reality it will get its data from node.js.

There are some challenges here. UAE (the original) is not async but ordinary, linear C code. JavaScript is async and may not return the data on exit of the method. So i suspect I will have to cache quite a lot. Perhaps as much as 1 megabyte backwards and forwards from the file-position. But getting data in there we will (and out), come hell or high water.

We can also drop a lot of sub code, like parts of the gayle interface. I found out this is the chip that deals with the IDE interface — so it has to be there, but it can also host memory expansions – but who the hell cares in 2017 in JavaScript about that. More than enough fun via standard chip/fast/rtg memory – so the odd bits can be removed.

So we got our work cut out for us. But hey.. there can only be one .. QUARTEX! And we bring you fire.

Ok. Lets do this!

Amibian + Smart pascal = A new beginning

March 25, 2017 Leave a comment

In the Amiga community there is a sub-group of people with an agenda. They hoard and collect every piece of hardware they can get their hand on – then sell them at absurd prices on ebay to enrich themselves. This is not only ruining the community at large, ensuring that ordinary users cannot get their hands on a plain, vanilla Amiga without having to fork out enough dough to buy a good car.

“We also have a working Facebook clone – but we’re not
going into competition with Mark Zuckerberg either”

Thankfully not everyone is like that. There are some very respected, very good people who buy old machines, restore them – and sell them at affordable prices. People that do this as a hobby or to make a little on the side. Nothing wrong with that. No, its people that try to sell you an A2000 for $3000, or that pimp out Vampire accelerator cards at 700€ a piece thats the problem.

As for the price sharks, well – this has to stop. And Gunnar’s Amibian distro has already given the Amiga scalpers a serious uppercut. Why buy an old Amiga when you can get a high-end A4000 experience with 4 times as much power for around $35? This is what Gunnar has made a reality – and he deserves a medal for his work!

And myself, Thomas and the others in our little band of brothers will pick up the fight and stand by Gunnar in his battle. A battle to make the Amiga affordable for ordinary human beings that love the platform.

Amiga as a service

Yesterday I did a little experiment. You know how Citrix and VMWare services work? In short, they are virtualization application servers. That means that they can create as many instances of Windows or Linux as they want – run your applications on it – and you can connect to your instance and use it. This is a big part of how cloud computing works.

While my little experiment is very humble, I am now streaming a WinUAE instance display from my basement server pc, just some old bucket of chips I use for debugging, directly into Amibian.js (!). It worked! I just created the world’s first Amiga application server. And it took me less than 30 minutes in Delphi !

Amibian, Amibian.js, appserver, what gives?

Let’s clear this up so you dont mix them:

Amibian. This is the original Linux distro made by Gunnar Kristjansson. It boots straight into UAE in full-screen. All it needs is a Raspberry PI, the Amiga rom files and your workbench or hard disk image. This is a purely native (machine code) solution.

Amibian.js – this is a JavaScript remake of AmigaOS that I’m building, with the look and feel of OS 4. It uses uae.js (also called SAE) to run 68k software directly in the browser. It is not a commercial product, but one of many demos that ship with Smart Mobile Studio. “Smart” is a compiler, editor and run-time library sold by The Smart Company AS. So Amibian.js is just a demo, just like Amos shipped with a ton of demo’s and Blitzbasic came with a whole disk full of examples.

Amiga application server (what I mentioned above) was just a 30 minute experiment I did yesterday after work. Again, just for fun.

I hope that clears up any confusion. Amibian.js is a purely JavaScript based desktop made in the spirit of the Amiga – it must not be confused with Amibian the Linux distro, which boots into UAE on a Raspberry PI. Nor is it an appserver – but rather, it can connect to an appserver if I want to.

genamiga

Generation Amiga post on Amibian.js earlier when I added some look & feel

With a bit of work, and if everything works as expected (I don’t see why not), I will upload both source and binaries to github together with Amibian.js.

There is only one clause: It cannot be used, recreated, included or distributed by Cloanto. Sorry guys, but the ROMS belong to the people, and until you release those into public domain, you wont get access to anything we make. Nothing personal, but pimping out roms and even having audacity to fork UAE and sell it as your own? You should be ashamed of yourself.

Are you in competition with FriendOS?

This question has popped up a couple of times the past two weeks. So I want to address that head on.

I make a product called Smart Mobile Studio. I do that with a group of well-known developers, and we have done so for many years now. The preliminary ideas were presented on my blog during the winter 2009, early 2010 and we started working (and blogging) after that. Smart Mobile Studio and it’s language, Smart Pascal (see Wikipedia article), takes object pascal (like freepascal or Delphi) and compiles to JavaScript rather than machine code.

smsamiga

The Smart compiler is due for OS4 once i get the A1222 in my hands

One of the examples that has shipped with Smart Mobile Studio, and also been available through a library called QTX, is something called Quartex Media Desktop. Which is an example of a NAS server front-end, a kiosk front-end (ticket ordering, cash machines etc) or just an intranet desktop where you centralize media and files. It is also node.js powered to deal with the back-end filesystem. This is now called amibian.js.

In other words – it has nothing to do with Friend software labs at all. In fact, I didn’t even know Friend existed until they approached me a few weeks ago.

15590846_10154070198815906_4207939673564686511_o

The Quartex Media desktop has been around for ages

Amibian.js is just an update of the Quartex Media Desktop example. It is not a commercial venture at all, but an example of how productive you can be with Smart pascal.

And it’s just one example out of more than a hundred that showcase different aspects of our run-time library. This example has been available since version 1.2 or 1.3 of Smart, so no, this is not me trying to reverse engineer FriendOS. Because I was doing this long before FriendOS even was presented. I have just added a windowing manager and made it look like OS4, which also happened before I had any contact with my buddies over at Friend Software Labs (why do you think they were interested in me).

16805402_1914020898827741_149245853_o

Early 2017 Linux bootloader by Gunnar

So, am I in competition with Friend? NO! I have absolutely no ambition, aspiration or intent for anything of the sorts. And should you be in doubt then let me break it down for you:

  • Hogne, Arne, David, Thomas, Francois and everyone at Friend Software Labs are friends of mine. I talk almost daily with David Pleasence who is a wonderful person and an inspiration for everyone who knows him.
  • Normal people don’t sneak around stabbing friends in the back. Plain and simple. That is not how I was raised, and such behavior is completely unacceptable.
  • Amibian.js is 110% pure Amiga oriented. The core of it has been a part of Smart for years now, and it has been freely available for anyone on google code and github.
  • For every change we have made to the Smart RTL, the media desktop example has been updated to reflect this. But ultimately it’s just one out of countless examples. We also have a working Facebook clone – but we’re not going into competition with Mark Zuckerberg for that matter.
  • People can invent the same things at the same time. Thats how reality works. There is a natural evolution of ideas, and great minds often think alike.

Why did you call it Amibian.js, it’s so confusing?

Well it’s a long story but to make it short. The first “boot into uae” thing was initially outlined by me (with the help of chips, the UAE4Arm maintainer). But I didn’t do it right because Linux has never really been my thing. So I just posted it on my retro-gaming blog and forgot all about it.

Gunnar picked this up and perfected it. He has worked weeks and months making Amibian into what it is today – together with Thomas, our spanish superhero /slash/ part-time dictator /slash/ minister of propaganda 🙂

We then started talking about making a new system. Not a new UAE, but something new and ground breaking. I proposed Smart Pascal, and we wondered how the Raspberry PI would run JavaScript performance wise. I then spent a couple of hours adding the icon layout grid and the windowing manager to our existing media desktop – and then fired up some HTML5 demos. Gunnar tested them under Chrome on the Raspberry PI — and voila, Amibian.js was born.

amidesk

And that is all there is to it. No drama, no hidden agendas – and no conspiracy.

I should also add that I do not work at Friend Software Labs, but we have excellent communication and I’m sure we will combine our forces on more than one software title in the future.

On a personal note I have more than a few titles I would like to port to FriendOS. One of my best sellers of all time is an invoice and credit application – which will be re-written in Smart Pascal (its presently a mix of Delphi and C++ builder code). The same program is also due to Amiga OS 4.1 whenever I get my A1222 (looking at you Trevor *smile*).

Well, I hope that clears up any misunderstanding regarding these very separate but superficially related topics. Amibian.js will remain 100% Amiga focused – that has been and remains our goal.

Ode to our childhood

Amibian is and will always be, an ode to the people who gave us such a great childhood. People like David Pleasence who was the face of Commodore in europe. A man who embody the friendliness of the Amiga with his very being. Probably one of the warmest and kindest people I can think of.

Francois Lionet, author of Amos Basic. The man who made me a programmer and that I cannot thank enough. And I know I’m not alone about learning from him.

Mark Sibly, the author of BlitzBasic, the man who taught me all those assembler tricks. A man that deserves to go down in the history books as one of the best programmers in history.

And above all – the people who made the Amiga itself; giants like Jay Miner, Dave Haynie, Carl Sassenrath, Dave Needle, RJ Michal (forgive me for not listing all of you. Your contributions will never be forgotten).

That is what Amibian.js is all about.

Patents and greed may have killed the actual code. But we are free to implement whatever we like from scratch. And when I’m done – your patents will be worthless..

 

Ghost of xmas past, or better known as the folder where old projects and abandoned ideas end up

December 8, 2016 Leave a comment

I think everyone has a folder where they stuff old units, test projects and well – all those ideas that seemed awesome at the time, but you either discovered that it was crap, not enough time, or just forgot all about it. It happens. What can I say.

Yeah, I have one of those as well. And I also have those duplicate folders. You know, that time when you decided to make backups for everything – but then you got a new PC and now you dont have a clue which version to keep? And when you finally sit down with beyondcompare to get this sorted, you have managed to add to both copies.

Well, needless to say I had a day I just went apeshit about a year ago and sorted everything. And whatever did not fit into my neat new code-folder(s) was mercilessly stuffed into the dark, obscure folder known only as “stuff” or “misc”.

My first basic compiler, awwwww how cute

Well its not all rubbish! I did find a special version of my kitchen-sink research IDE, the IDE I use when trying out new code for DWScript, PaxCompiler and FreePascal. I think it has 4 compilers in it or something, yet its under 30 megabytes in size! Pretty neat 🙂

Visual Basic to Smart Pascal, awww.... just want to snuggle it :)

Visual Basic to Smart Pascal, awww…. just want to snuggle it 🙂

It also features a unified file-system! You can mount packages, FTP folders, network paths or local folders – the IDE could not care less, it is completely abstracted from the actual filesystem and relates only to symbols and identifiers. The actual storage is dealt with by the filesource classes.

Package, ftp, socket, local folder - the IDE could not care less

Package, ftp, socket, local folder – the IDE could not care less

The Basic dialect here is essentially classical Visual Basic. The way you would write classes and deal with instances before dot net came along and forced all languages to look the same. I kinda like old visual basic, it had soul. It was completely useless except for scripting, and Delphi made it look like a joke – but as far as basic dialects go, it’s probably one of the cleanest and easiest to read.

DWScript, QTX Pascal (a custom fork of DWScript with a lot of cool features, some that Eric has added afterwards), Basic - and even freepascal! Thats the mother of all kitchen sinks!

DWScript, QTX Pascal (a custom fork of DWScript with a lot of cool features, some that Eric has added afterwards), Basic – and even freepascal! Thats the mother of all kitchen sinks!

The loss of QTX Pascal is a bit sad. I spent a couple of months doing my own fork of DWScript. Most of the features I added have now been included (although in another form) by Eric. But the codegen produced faster javascript. One of the things I spent some time on was optimization. Like getting rid of “variable = 0” type initialization if an assignment followed. I also added BeforeDestruction() and AfterConstruction() calls. This made an RTL a lot easier to write, but also more overhead. I was about to do conditional testing (so these would only be called if you actually used them) when I had to stop and work on Smart Mobile Studio again.

N++, exotic and functional!

This was one of my favorite language research projects. And dare I say, it is way ahead of it’s time. The idea is simple: with the future of computing being distributed, cloud based and powered by a multitude of computing modules from the US to South Africa — what would a programming language look like if built for 100% asyncronous, distributed execution?

Consider for instance, the execution of a procedure. Simple right? Well, in a truly distributed system – that procedure could execute anywhere. The whole point here is to utilize the combined power of each module (pc); which means a tasks could execute on different computers. A procedure could also be split up by the runtime environment and once again, be executed all over the place – with the dispatcher keeping track of when it’s done and whatever the result was (if any).

Only a spesific and generated on-demand “runtime context” would follow the task. This would contain things like variables it needs to execute, public symbols, basicaly just enough for the code to run successfully. This also includes resource forks (a term i used for accessing resources regardless of where they may be on the network).

The language design was actually done using Smart Mobile Studio (you can read more about the initial ideas here), and the first test modules ran on node.js (so no fancy graphics available sadly).

But it was incredibly fun to play with! But also time consuming to design a language for an execution model that havent really been invented yet. You can read more about some of my ideas for the execution model here.

I dont even think this execution model is out of the MIT labs yet — but it will once quantum compute models become commercially available (click here for a Berkley University published introduction to the basic computational models used in quantum information theory).

An N++ procedure (actually a microservice that consumed a WSDL service and calls it) looks like this:

program("service_test") {

  handshake {

    input {
      /* Consume WSDL Web-Service Endpoint */
      service1 @ service[URI:"http://www.test.com/SOAP/myService/WSDL/",
                 serviceType:SoapService];
    }

    output {
        myProcess @ process("self");
        stdio @ pipe("stdout");
      }
  }

  /* Execute RPC call */
  execute (stdio,service1)  {
    stdio:writeln("Calling webservice");
    execute (*)  {
      var int32 result = 0;
      set result = service1:getUserId("quartex","secret");
      stdio:writelnF("UserID on server={0}", result);
    } fail (e) {
      stdio.writelnF("Calling soap service failed: {0}",e);
      proceed;
    }
  }

  /* Exit code for process */
  set myProcess:exitCode = 0;
}

RML, Reduced markup language parser and runtime

Dont you just hate it when you lose the source-file for a programming language you made in a day, so now you dont have a clue what the syntax really was? I know, I know, but I honestly dont have the syntax or examples. It does with a disk ages ago.

RML was written to simplify generating HTML documents. There are a few things that you can optimize when it comes to HTML. First of all you know that the browser will read the document from top and downwards. This might not mean much at first glance, but it actually means that a language doesnt need the abillity to call emitter-code that has already executed (above). Think of it as being able to omit calls to procedure behind you in the source. That would be a mess for a language like delphi – but it doesnt impact a language that execute linear without any forms of jumps!

I seem to remember it looked something like this:

createobject(TRMLObject)
{
   writestr("testing");
   if(a=12)
   {
       readstr("mer testing");
   }
}
createobject(TRMLHeading)
{
   settitle("testing second object");
   if(a=12)
   {
       writestr("heading object successfully done");
   }
}
createobject(TRMLObject)
{
   writestr("testing third object");
   if(a=12)
   {
       readstr("mer testing from da third object");
   }
}

The idea is ofcourse that you have some pre-defined objects, a bit like custom-controls in Smart Pascal, and you create and populate these in a top-down fashion.
Once defined, generating quite advanced HTML documents can be done extremely fast. Much faster than ASP, DWS or php since no jumps are involved.

Here is how you compile and run a small debug session (drop a memo and button on a form first):

  var
    FContext: TRMLParseContext;
    FText:  String;
  begin
    FText:= 'createobject(TRMLObject)'
    +       '{'
    +       '   writestr("testing");'
    +       '   if(a=12)'
    +       '   {'
    +       '       readstr("mer testing");'
    +       '   }'
    +       '}'
    +       'createobject(TRMLHeading)'
    +       '{'
    +       '   settitle("testing second object");'
    +       '   if(a=12)'
    +       '   {'
    +       '       writestr("heading object successfully done");'
    +       '   }'
    +       '}'
    +       'createobject(TRMLObject)'
    +       '{'
    +       '   writestr("testing third object");'
    +       '   if(a=12)'
    +       '   {'
    +       '       readstr("mer testing from da third object");'
    +       '   }'
    +       '}';

    // double the code, just to get some payload
    FText := FText + FText;

    memo1.text:='';

    If RMLAllocContext(FContext,FText,[TRMLObject,TRMLHeading]) then
    Begin
      try
        if RMLCompile(FContext) then
        Begin
          caption:='Compilation was a success';
          If RMLExecute(FContext) then
          Begin
            Caption:='Executed successfully';
            memo1.text:=FContext.pcOutput;
          end;
        end else
        caption:=Format('compilation failed [%s]',[FContext.pcError]);
      finally
        RMLDisposeContext(FContext);
      end;
    end;

And here is the full code for the parser and runtime (still a few tidbits to work on, but its childs play). If you can make sense of it, knock yourself out 🙂

  unit rml;

  interface

  uses sysutils, classes, dialogs;

  type

  TRMLObject  = Class;

  TRMLClass     = Class of TRMLObject;
  TRMLOperator  = (opLess,opMore,opEquals,opNotEquals);
  TRMLDataType  = (daString,daNumber,daBoolean,daCustom);
  TRMLEntryType = (etAssignment,etMethod,etRepeat,etCreate,etIf);
  TRMLObjProc   = Function (Const EntryData:Pointer):Boolean of Object;

  (* represents a single condition in an IF statement *)
  PRMLCondition = ^TRMLCondition;
  TRMLCondition = Packed Record
    coSource:   String;
    coTarget:   String;
    coOperator: TRMLOperator;
  End;

  (* represents a parsed code declaration *)
  PRMLEntryDeclaration = ^TRMLEntryDeclaration;
  TRMLEntryDeclaration = Packed Record
    cxToken:      String;
    cxCondition:  String;
  End;

  (* represents a compiled parameter *)
  PRMLParameter = ^TRMLParameter;
  TRMLParameter = Packed Record
    prType:   TRMLDataType;
    prValue:  Pointer;
    prSize:   Integer;
  End;

  (* represents a compiled code entry *)
  PRMLEntryData = ^TRMLEntryData;
  TRMLEntryData = Packed Record
    edtype:         TRMLEntryType;
    edDeclaration:  TRMLEntryDeclaration;
    edObject:       TRMLObject;
    edParent:       PRMLEntryData;
    edMethod:       TRMLObjProc;
    edContext:      Pointer;
    edConditions:   Array of PRMLCondition;
    edParameters:   Array of PRMLParameter;
    edSubEntries:   Array of PRMLEntryData;
  End;

  PRMLParseContext = ^TRMLParseContext;
  TRMLParseContext = Packed Record
    pcSignature:  Integer;
    pcCol:        Integer;
    pcRow:        Integer;
    pcPos:        Integer;
    pcLen:        Integer;
    pcData:       String;
    pcError:      String;
    pcOutput:     String;
    pcRoot:       TRMLEntryData;
    pcClasses:    Array of TRMLClass;
  End;

  TRMLReadProc  = Function  (Var OutData;var Bytes:Integer):Boolean of Object;
  TRMLWriteProc = Function  (Var InData;Const Bytes:Integer):Boolean of Object;
  TRMLProcEntry = Function  (Const Entry:PRMLEntryData):Boolean of Object;

  PRMLObjectIndex = ^TRMLObjectIndex;
  TRMLObjectIndex = Packed Record
    oiMethods:    Array of record
                    omName:   String;
                    omSyntax: Array of TRMLDataType;
                    omEntry:  TRMLProcEntry;
                  end;
    oiProperties: Array of record
                    oiName:   String;
                    oiRead:   TRMLReadProc;
                    oiWrite:  TRMLWriteProc;
                    oiType:   TRMLDataType;
                  end;
  End;

  TRMLObject = Class(TObject)
  Private
    FIndexData: TRMLObjectIndex;
    FIndexPTR:  PRMLObjectIndex;
  Private
    Function    DoWriteStr(Const Entry:PRMLEntryData):Boolean;
    Function    DoReadStr(Const Entry:PRMLEntryData):Boolean;
  protected
    Procedure   Output(Const Context:PRMLParseContext;Const Value:String);
    Procedure   RegisterProperty(Const Name:String;Const DataType:TRMLDataType;
                Const _Read:TRMLReadProc;Const _Write:TRMLWriteProc);
    Procedure   RegisterMethod(Const Name:String;
                Const Syntax: Array of TRMLDataType;
                Const Entry: TRMLProcEntry);
  Public
    Property    ObjectIndex:PRMLObjectIndex read FIndexPTR;
    Constructor Create;virtual;
  End;

  TRMLHeading = Class(TRMLObject)
  Private
    FTitle:     String;
    Function    DoSetTitle(Const Entry:PRMLEntryData):Boolean;
    Function    DoReadTitle(Var OutData;var Bytes:Integer):Boolean;
    Function    DoWriteTitle(Var InData;Const Bytes:Integer):Boolean;
  Public
    property    Title:String read FTitle write FTitle;
    Constructor Create;override;
  End;

  Function  RMLAllocContext(var Context:TRMLParseContext;
            Const Source:String;Const ClsBase:Array of TRMLClass):Boolean;
  Function  RMLDisposeContext(Var Context:TRMLParseContext):Boolean;
  Function  RMLCompile(Var Context:TRMLParseContext):Boolean;
  Function  RMLExecute(Const Context:TRMLParseContext):Boolean;
  Function  RMLParseEntry(Value:String;
            var Declaration:TRMLEntryDeclaration;var Error:String):Boolean;

  implementation

  //###########################################################################
  // TRMLHeading
  //###########################################################################

  Constructor TRMLHeading.Create;
  Begin
    inherited;
    RegisterMethod('settitle',[daString],DoSetTitle);
    RegisterProperty('title',daString,DoReadTitle,DoWriteTitle);
  end;

  Function TRMLHeading.DoSetTitle(Const Entry:PRMLEntryData):Boolean;
  var
    FTemp:  String;
  Begin
    result:=length(Entry^.edParameters)>0;
    If result and (Entry^.edParameters[0].prType=daString) then
    DoWriteTitle(Entry^.edParameters[0].prValue^,
    Entry^.edParameters[0].prSize);
  end;

  Function TRMLHeading.DoReadTitle(Var OutData;var Bytes:Integer):Boolean;
  Begin
    Bytes:=Length(FTitle);
    If Bytes>0 then
    move(FTitle[1],outData,Bytes);
  end;

  Function TRMLHeading.DoWriteTitle(Var InData;Const Bytes:Integer):Boolean;
  Begin
    SetLength(FTitle,bytes);
    if Bytes>0 then
    move(inData,FTitle[1],Bytes);
  end;

  //###########################################################################
  // TRMLObject
  //###########################################################################

  Constructor TRMLObject.Create;
  begin
    inherited;
    FIndexPTR:=@FIndexData;
    RegisterMethod('writestr',[daString],DoWriteStr);
    RegisterMethod('readstr',[daString],DoReadStr);
  end;

  Procedure TRMLObject.Output(Const Context:PRMLParseContext;
            Const Value:String);
  Begin
    Context^.pcOutput:=Context^.pcOutput + Value;
  end;

  Function TRMLObject.DoReadStr(Const Entry:PRMLEntryData):Boolean;
  var
    FText:  String;
  Begin
    result:=True;
    With Entry^ do
    Begin
      FText:='Token: ' + edDeclaration.cxToken + #13#10;
      FText:=FText + 'Condition: ' + edDeclaration.cxCondition + #13#10;
      FText:=FText + #13#10;
      Output(edContext,FText);
    end;
  end;

  Function TRMLObject.DoWriteStr(Const Entry:PRMLEntryData):Boolean;
  var
    FText:  String;
  Begin
    result:=True;
    With Entry^ do
    Begin
      FText:='Token: ' + edDeclaration.cxToken + #13#10;
      FText:=FText + 'Condition: ' + edDeclaration.cxCondition + #13#10;
      FText:=FText + #13#10;
      Output(edContext,FText);
    end;
  end;

  Procedure   TRMLObject.RegisterProperty(Const Name:String;
              Const DataType:TRMLDataType;
              Const _Read:TRMLReadProc;Const _Write:TRMLWriteProc);
  var
    FCount: Integer;
  Begin
    //FCount:=high(FIndexData.oiProperties) - Low(FIndexData.oiProperties) + 1;
    FCount:=Length(FIndexData.oiProperties);
    SetLength(FIndexData.oiProperties,FCount+1);
    FIndexData.oiProperties[FCount].oiName:=Name;
    FIndexData.oiProperties[FCount].oiRead:=_Read;
    FIndexData.oiProperties[FCount].oiWrite:=_Write;
    FIndexData.oiProperties[FCount].oiType:=DataType;
  end;

  Procedure   TRMLObject.RegisterMethod(Const Name:String;
              Const Syntax: Array of TRMLDataType;
              Const Entry: TRMLProcEntry);
  var
    FCount: Integer;
    FTemp:  Integer;
  Begin
    //FCount:=high(FIndexData.oiMethods) - Low(FIndexData.oiMethods) + 1;
    FCount:=Length(FIndexData.oiMethods);
    SetLength(FIndexData.oiMethods,FCount+1);
    FIndexData.oiMethods[FCount].omName:=Name;
    FIndexData.oiMethods[FCount].omEntry:=Entry;

    //FTemp:=high(Syntax) - Low(Syntax) + 1;
    FTemp:=Length(Syntax);
    If FTemp>0 then
    Begin
      SetLength(FIndexData.oiMethods[FCount].omSyntax,FTemp);
      for FTemp:=low(syntax) to high(syntax) do
      FIndexData.oiMethods[FCount].omSyntax[FTemp]:=Syntax[FTemp];
    end;
  end;

  //###########################################################################
  // RML util methods
  //###########################################################################

  Function RMLContainChars(Const Value:String;const Chars:String):Boolean;
  var
    x:  Integer;
  Begin
    result:=True;
    for x:=1 to length(chars) do
    Begin
      if pos(chars[x],Value)<1 then
      Begin
        result:=False;
        Break;
      end;
    end;
  end;

  Function  RMLScanFor(const Value:String;Const Target:CHAR;
            Const Breakers:String;var Len:Integer):Boolean;
  var
    xpos: Integer;
  Begin
    result:=False;
    Len:=-1;
    xpos:=1;
    while xpos<=Length(Value) do     Begin       If Value[xpos]=Target then       Begin         Len:=xpos-1;         Result:=True;         Break;       end else       Begin         if pos(Value[xpos],Breakers)>0 then
        Break;
      end;
      inc(xpos);
    end;
  end;

  Function RMLisNumber(Const Value:String):Boolean;
  const
    CHARSET = '0123456789';
  var
    x:  Integer;
  Begin
    Result:=True;
    for x:=1 to length(Value) do
    Begin
      if pos(Value[x],CHARSET)<1 then
      Begin
        result:=False;
        Break;
      end;
    end;
  end;

  Function RMLIsBoolean(Const Value:String):Boolean;
  var
    FTemp:  String;
  Begin
    FTemp:=lowercase(trim(Value));
    result:=(FTemp='false') or (FTemp='true');
  end;

  Function RMLisString(Const Value:String):Boolean;
  var
    x:      Integer;
    FLeft:  Integer;
  Begin
    result:=False;
    FLeft:=0;
    (* check left side *)
    for x:=1 to length(Value) do
    Begin
      if Value[x]='"' then
      Begin
        FLeft:=x;
        Result:=True;
        Break;
      end else
      if Value[x]<>#32 then
      Break;
    end;
    (* check right side *)
    If result then
    Begin
      for x:=Length(Value) downto 1 do
      Begin
        if Value[x]='"' then
        Begin
          If x>FLeft then
          Break else
          Begin
            Result:=False;
            Break;
          end;
        end else
        if Value[x]<>#32 then
        Break;
      end;
    end;
  end;

  Function  RMLParseEntry(Value:String;
            var Declaration:TRMLEntryDeclaration;
            var Error:String):Boolean;
  var
    xpos: Integer;
  Begin
    fillchar(Declaration,SizeOf(Declaration),0);
    Result:=RMLContainChars(value,'()');
    if Result then
    Begin
      Result:=RMLScanFor(value,'(',')',xpos);
      if result then
      Begin
        Declaration.cxToken:=trim(copy(value,1,xpos));
        delete(value,1,xpos+1);
        Result:=RMLScanFor(value,')','(',xpos);
        if result then
        Begin
          Value:=TrimRight(Value);
          Result:=xpos=(length(value)-1);
          if result then
          Declaration.cxCondition:=trim(Copy(Value,1,xpos));
        end;
      end;
    end;
    If not Result then
    Error:='Invalid entry <' + value + '>';
  end;

  Function  RMLAllocContext(var Context:TRMLParseContext;
            Const Source:String;Const ClsBase:Array of TRMLClass):Boolean;
  var
    FCount: Integer;
  Begin
    If Context.pcSignature=SizeOf(Context) then
    RMLDisposeContext(Context);

    fillchar(Context,SizeOf(Context),#0);
    Context.pcSignature:=SizeOf(Context);
    Context.pcLen:=Length(Source);
    Context.pcData:=Source;

    FCount:=High(clsBase) - low(clsBase)+1;
    If FCount>0 then
    Begin
      SetLength(Context.pcClasses,FCount);
      for FCount:=Low(clsBase) to high(clsBase) do
      Context.pcClasses[FCount]:=clsBase[FCount];
    end;

    result:=True;
  end;

  Procedure RMLDisposeEntryData(Const Data:PRMLEntryData);
  var
    FTemp:  Integer;
    x:      Integer;
  Begin
    (* dispose of condition data *)
    FTemp:=length(Data^.edConditions);
    While FTemp>0 do
    Begin
      Dispose(Data^.edConditions[FTemp-1]);
      dec(FTemp);
    end;
    SetLength(Data^.edConditions,0);

    (* dispose of parameter data *)
    FTemp:=length(Data^.edParameters);
    While FTemp>0 do
    Begin
      If length(Data^.edParameters)>0 then
      Begin
        for x:=Low(Data^.edParameters) to high(Data^.edParameters) do
        If Data^.edParameters[x]^.prSize>0 then
        FreeMem(Data^.edParameters[x]^.prValue);
      end;
      Dispose(Data^.edParameters[FTemp-1]);
      dec(FTemp);
    end;
    SetLength(Data^.edParameters,0);

    (* dispose of sub entries *)
    //Ftemp:=High(Data^.edSubEntries)-Low(Data^.edSubEntries)+1;
    FTemp:=Length(Data^.edSubEntries);
    While FTemp>0 do
    Begin
      RMLDisposeEntryData(Data^.edSubEntries[FTemp-1]);
      Dec(FTemp);
    end;
    SetLength(Data^.edSubEntries,0);

    (* dispose of script object *)
    If  not (Data^.edtype in [etIf,etRepeat,etMethod])
    and (Data^.edObject<>NIL) then
    Data^.edObject.free;

    (* dispose of entry *)
    Dispose(Data);
  end;

  Function  RMLDisposeContext(Var Context:TRMLParseContext):Boolean;
  var
    FCount: Integer;
  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If Result then
    Begin
      Context.pcSignature:=0;
      Context.pcData:='';
      Context.pcError:='';
      FCount:=Length(Context.pcRoot.edSubEntries);
      //FCount:=High(Context.pcRoot.edSubEntries)
      //- Low(Context.pcRoot.edSubEntries) + 1;
      While FCount>0 do
      Begin
        RMLDisposeEntryData(Context.pcRoot.edSubEntries[FCount-1]);
        dec(FCount);
      end;
      SetLength(Context.pcRoot.edSubEntries,0);
    end;
  end;

  //###########################################################################
  // RML core methods
  //###########################################################################

  Function  RMLImplements(Const MethodName:String;
            Const obj:TRMLObject):Boolean;
  var
    FTable: PRMLObjectIndex;
    FCount: Integer;
  Begin
    Result:=Obj<>NIL;
    If result then
    Begin
      (* get object inex *)
      FTable:=Obj.ObjectIndex;
      //FCount:=High(FTable^.oiMethods) - low(FTable^.oiMethods) + 1;
      FCount:=Length(FTable^.oiMethods);
      Result:=FCount>0;
      If Result then
      Begin
        for FCount:=low(FTable^.oiMethods) to high(FTable^.oiMethods) do
        Begin
          Result:=FTable^.oiMethods[FCount].omName=MethodName;
          If Result then
          Break;
        end;
      end;
    end;
  end;

  Function  RMLGetMethodEntry(Const MethodName:String;
            Const obj:TRMLObject;var outEntry:TRMLObjProc):Boolean;
  var
    FTable: PRMLObjectIndex;
    FCount: Integer;
  Begin
    Result:=Obj<>NIL;
    If result then
    Begin
      (* get object inex *)
      FTable:=Obj.ObjectIndex;
      //FCount:=High(FTable^.oiMethods) - low(FTable^.oiMethods) + 1;
      FCount:=Length(FTable^.oiMethods);
      Result:=FCount>0;
      If Result then
      Begin
        for FCount:=low(FTable^.oiMethods) to high(FTable^.oiMethods) do
        Begin
          Result:=FTable^.oiMethods[FCount].omName=MethodName;
          If Result then
          Begin
            outEntry:=TRMLObjProc(FTable^.oiMethods[FCount].omEntry);
            Break;
          end;
        end;
      end;
    end;
  end;

  Function  RMLCreateObject(var Context:TRMLParseContext;
            Const Objname:String;var outObject:TRMLObject;
            var Error:String):Boolean;
  var
    FCount: Integer;
  Begin
    FCount:=High(Context.pcClasses) - Low(Context.pcClasses) + 1;
    Result:=FCount>0;
    if Result then
    Begin
      For FCount:=Low(Context.pcClasses) to high(Context.pcClasses) do
      Begin
        Result:=lowercase(Context.pcClasses[FCount].ClassName)=lowercase(objName);
        If result then
        Begin
          outObject:=Context.pcClasses[FCount].Create;
          Break;
        end;
      end;
    end;
    If not Result then
    Error:='Unknown class <' + Objname + '>';
  end;

  Function  RMLAddEntry(var Context:TRMLParseContext;
            Var Declaration:TRMLEntryDeclaration;
            Root:PRMLEntryData;var NewEntry:PRMLEntryData;
            var Error:String):Boolean;
  var
    FCount: Integer;
    x:      Integer;
    FTemp:  String;
    FLen:   Integer;
    FPar:   PRMLParameter;
  Begin
    Result:=Root<>NIL;
    If result then
    Begin
      (* create new entry *)
      new(NewEntry);

      (* Reset entry record *)
      NewEntry^.edType:=etAssignment;
      NewEntry^.edObject:=NIL;
      NewEntry^.edMethod:=NIL;
      SetLength(NewEntry^.edConditions,0);
      SetLength(NewEntry^.edParameters,0);
      SetLength(NewEntry^.edSubEntries,0);

      (* Set basic values *)
      NewEntry^.edParent:=Root;
      NewEntry^.edContext:=@Context;
      newEntry^.edDeclaration:=Declaration;

      (* insert entry into parent *)
      FCount:=Length(Root^.edSubEntries);
      SetLength(Root^.edSubEntries,FCount+1);
      Root^.edSubEntries[FCount]:=NewEntry;

      (* tokenize *)
      If declaration.cxToken='createobject' then
      Begin
        NewEntry^.edtype:=etCreate;
        Result:=RMLCreateObject
          (
          Context,declaration.cxCondition,
          NewEntry^.edObject,Error
          );
      end else

      if declaration.cxToken='if' then
      Begin
        NewEntry^.edtype:=etIF;
        NewEntry^.edObject:=NewEntry^.edParent.edObject;
      end else

      if declaration.cxToken='repeat' then
      NewEntry^.edtype:=etRepeat else

      Begin
        (* method call? Make sure entry object supports this *)
        Result:=NewEntry^.edParent.edObject<>NIL;
        If Result then
        Begin
          (* check if object supports the method name *)
          Result:=RMLImplements(declaration.cxToken,NewEntry^.edParent.edObject);
          If Result then
          Begin
            (* Query object for method entry *)
            Result:=RMLGetMethodEntry
              (
              declaration.cxToken,
              NewEntry^.edParent.edObject,
              NewEntry^.edMethod
              );

            If result then
            Begin
              NewEntry^.edtype:=etMethod;
              NewEntry^.edObject:=NewEntry^.edParent.edObject;

              (* now parse the parameter conditions *)
              x:=0;
              While x<Length(declaration.cxCondition) do               Begin                 inc(x);                 If (declaration.cxCondition[x]=',')                 or (x=Length(declaration.cxCondition)) then                 Begin                   If x=Length(declaration.cxCondition) then                   FTemp:=FTemp + declaration.cxCondition[x];                   FTemp:=trim(FTemp);                   If length(FTemp)>0 then
                  Begin
                    (* create a new parameter *)
                    FLen:=length(NewEntry^.edParameters);
                    setlength(NewEntry^.edParameters,FLen+1);

                    New(FPar);

                    If RMLIsString(FTemp) then
                    FPar^.prType:=daString else
                    if RMLIsNumber(FTemp) then
                    FPar^.prType:=daNumber else
                    if RMLIsBoolean(FTemp) then
                    FPar^.prType:=daBoolean else
                    FPar^.prType:=daCustom;

                    Case FPar^.prType of
                    daString:
                      Begin
                        Delete(FTemp,1,1);
                        Delete(FTemp,length(FTemp),1);
                        FPar^.prSize:=Length(FTemp);
                        FPar^.prValue:=AllocMem(FPar^.prSize);
                        move(FTemp[1],FPar^.prValue^,FPar^.prSize);
                      end;
                    daNumber:
                      Begin
                        FPar^.prSize:=SizeOf(Integer);
                        FPar^.prValue:=AllocMem(FPar^.prSize);
                        PInteger(FPar^.prValue)^:=StrToInt(FTemp);
                      end;
                    daBoolean:
                      Begin
                      end;
                    daCustom:
                      Begin
                      end;
                    end;

                    NewEntry^.edParameters[FLen]:=FPar;
                    FTemp:='';
                  end else
                  Begin
                    //Invalid parameter error
                  end;
                end else
                FTemp:=FTemp + declaration.cxCondition[x]
              end;

              {
              Validate parameter datatypes here!
              If  (Length(NewEntry^.edParameters)>0) then
              Begin
                for x:=Low(NewEntry^.edParameters) to
                high(NewEntry^.edParameters) do
                Begin
                  newEntry^.edObject.
                end;
              end;  }

            end;

          end else
          Begin
            // property assignment test here
          end;
        end;
      end;

      (* Failed to tokenize? *)
      If not Result then
      Begin
        (* dispose of entry data *)
        Dispose(NewEntry);
        NewEntry:=NIL;
        SetLength(Root^.edSubEntries,FCount);
        Context.pcError:=Format('Invalid token "%s"',[declaration.cxToken]);
      end;

    end else
    Error:='AddEntry failed, root can not be NIL';
  end;

  Function  RMLParseObject(Var Context:TRMLParseContext;
            Const Root:PRMLEntryData):Boolean;
  var
    FChar:        Char;
    FTemp:        String;
    FDeclaration: TRMLEntryDeclaration;
    FNewEntry:    PRMLEntryData;
  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If result then
    Begin
      (* update cursor *)
      inc(Context.pcPos);
      inc(Context.pcCol);

      while Result and (Context.pcPos<Context.pcLen) do
      Begin
        FChar:=Context.pcData[Context.pcPos];
        Case FCHAR of
        #13:
          Begin
            inc(Context.pcRow);
            inc(Context.pcPos);
            Context.pcCol:=0;
            Continue;
          end;
        ';':
          Begin
            Result:=RMLParseEntry(trim(FTemp),FDeclaration,Context.pcError);
            If result then
            Result:=RMLAddEntry
              (
                Context,
                FDeclaration,
                root,
                FNewEntry,
                Context.pcError
              );

            If Result then
            Begin
              inc(Context.pcPos);
              inc(Context.pcCol);

              If FNewEntry^.edtype=etIF then
              Result:=RMLParseObject(Context,FNewEntry);
            end;
            FTemp:='';
          end;
        '{':
          Begin
            Result:=RMLParseEntry(FTemp,FDeclaration,Context.pcError);
            If Result then
            Begin
              Result:=FDeclaration.cxToken='if';
              If result then
              Begin
                Result:=RMLAddEntry
                  (
                  Context,
                  FDeclaration,
                  Root,
                  FNewEntry,
                  Context.pcError
                  );
                If Result then
                Result:=RMLParseObject(Context,FNewEntry);
              end;
            end;
            FTemp:='';
          end;
        '}':
          Begin
            inc(Context.pcCol);
            inc(Context.pcPos);
            Break;
          end;
        else
          Begin
            FTemp:=FTemp + FChar;
            inc(Context.pcCol);
            inc(Context.pcPos);
          end;
        end;
      end;
    end;
  end;

  Function RMLExecute(Const Context:TRMLParseContext):Boolean;

    Function RunEntry(Const Item:PRMLEntryData):Boolean;
    var
      FSubCount:  Integer;
      x:          Integer;
    Begin
      result:=True;

      Case Item^.edtype of
      etCreate,
      etAssignment:
        Begin

          FSubCount:=Length(Item^.edSubEntries);
          for x:=1 to FSubCount do
          Begin
            result:=RunEntry(Item^.edSubEntries[x-1]);
            If not result then
            break;
          end;

        end;
      etMethod:
        Begin
          {result:=RMLGetMethodEntry(Item^.edDeclaration.cxToken,
          Item^.edObject,FEntry);
          If result then
          result:=FEntry(Item); }
          Result:=TRMLProcEntry(Item^.edMethod)(Item);
        end;
      etRepeat:
        Begin
          //FSubCount:=Length(Item^.edSubEntries);
        end;
      etIf:
        Begin
          FSubCount:=Length(Item^.edSubEntries);
          for x:=1 to FSubCount do
          RunEntry(Item^.edSubEntries[x-1]);
        end;
      end;
    End;

  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If result then
    Begin
      result:=length(Context.pcError)<1;
      If result then
      result:=RunEntry(@Context.pcRoot);
    end;
  end;

  Function RMLCompile(Var Context:TRMLParseContext):Boolean;
  var
    FChar:        Char;
    FTemp:        String;
    FDeclaration: TRMLEntryDeclaration;
    FNewEntry:    PRMLEntryData;
  Begin
    Result:=Context.pcSignature=SizeOf(Context);
    If result then
    Begin
      Context.pcCol:=0;
      Context.pcRow:=0;
      Context.pcPos:=1;
      Context.pcError:='';

      while Result and (Context.pcPos<Context.pcLen) do
      Begin
        FChar:=Context.pcData[Context.pcPos];
        Case FCHAR of
        #13:
          Begin
            inc(Context.pcRow);
            inc(Context.pcPos);
            Context.pcCol:=0;
            Continue;
          end;
        '{':
          Begin
            Result:=RMLParseEntry(FTemp,FDeclaration,Context.pcError);
            If Result then
            Begin
              Result:=FDeclaration.cxToken='createobject';
              If result then
              Begin
                Result:=RMLAddEntry
                  (
                  Context,
                  FDeclaration,
                  @Context.pcRoot,
                  FNewEntry,
                  Context.pcError
                  );
                If Result then
                Result:=RMLParseObject(Context,FNewEntry);
              end;
            end;
            FTemp:='';
          end;
        '}':
          Begin
          end;
        else
          Begin
            FTemp:=FTemp + FChar;
            inc(Context.pcCol);
            inc(Context.pcPos);
          end;
        end;
      end;
    end;
  end;

  end.

Why C# coders should shut up about Delphi

October 18, 2016 Comments off

EDIT: Some 3 years after this satire post was published, a website called “.NetRocks” decided to make a number out of it. The satire clearly went over their heads, and the facts I outline in the post was met with ridicule. I’m a bit disappointed with .NetRocks, because this could have been a great opportunity for them to learn more about modern object-pascal development and it’s ecosystems. They seem genuinely baffled that Delphi and Object-Pascal in general was being used in 2016 (and 2019), and their behavior and response demonstrates the very arrogance and ignorance we have come to expect from “brogrammers” of their type.

I must admit that I’m somewhat shocked that a massive show like .NetRocks is utterly ignorant of the fact that Object-Pascal as a language has millions of users around the world. You have compilers like Oxygene from RemObjects that targets .Net, Java, x86, ARM and WebAssembly. So their whole podcast on “how to port Delphi applications to C#” is absurd, because that’s not how you migrate code from Delphi to .Net. The Elements ecosystem, which covers C#, Pascal, Java, Swift and GoLang installs straight into Visual Studio and adds Object-Pascal to the environment. You can also convert source-code between Pascal and C# through automation. Manually going about such tasks is amateurish to say the least, but I honestly didn’t expect much when they open their podcast with verbal abuse.

vspas

While .NetRocks is manually porting pascal to C#, the rest of us use the Elements compiler and just compile the code to .Net Assemblies. You can also convert the code to any of the 5 languages Elements support. I have to wonder what rock .NetRocks lives under to be oblivious to this

Another point .NetRocks seem to have missed, is that Delphi has been aggressively optimized since Embarcadero took over from Borland. In 2016 Object Pascal ranked as #14 [2.164 score] on the Tiobi index over most popular programming languages [globally], just nine points below C#. So the most cost effective way of getting your legacy Delphi application modernized, is to just buy Elements from RemObjects, or upgrade your Delphi version to this century.

If they had bothered to check they would also have learned that I manage some 16+ developer groups on social media, with roughly 50.000 developers in total. I have also worked for Embarcadero that makes Delphi, so my statements while whimsical and satirical, were based on facts. I work with Fortune 500 companies in the US, as well as large companies around Europe, that all rely on Object-Pascal for their daily operations. Heck, even the Walt Disney company wanted to get rid of .Net in favour of Object Pascal in 2018. Turns out JIT is a disaster when coding against hardware interrupts and “old school” automation. Who could have guessed? (that was satire btw).

vsdelphi

Some people seem to think that Delphi ended with version 7 back in  the 90s. Delphi has been under heavy development since Embarcadero took over for Borland. .NetRocks doesnt even mention that buying a modern version of Delphi is an option companies should consider. And if they absolutely have to migrate to C#, then Elements is what professionals use.

On the whole, It’s alarming that the guys at .NetRocks seem clueless to the tools professional developers use for specialized work. Porting a Delphi codebase by hand its, well, amateurish at best. They might as well do a podcast on how to use ms-paint to create art, because Adobe Photoshop or Illustrator is not what people use (that was satire btw).

I am also pleased however, because .NetRocks unwittingly demonstrated my point to perfection 🙂 Instead of learning more about Anders Hejlsberg’s journey and life before he joined Microsoft, how the .Net class hierarchy is heavily influenced by the Delphi VCL framework, or how the concepts that ended up as .Net were in fact prototyped and experimented with earlier — they jumped straight to ridicule. Of a satire post (lol!).

You can click here for my full reply. And seriously, how it’s possible to mistake this old post as anything but heavy satire is beyond me. But I am thankful to .NetRocks for demonstrating my point. I could not have asked for a better example of arrogance and ignorance 🙂 Rock on!


 

13507225_10209662785317165_99874043994628045_n

When in Rome

Those that follow my blog or know me personally – also know that I don’t go out of my way to be confrontational or disrespectful. I try my best to follow the middle-path, to see positive in all things. But sometimes you face a level of stupid that begs a response. A verbal one. And this is one of those cases.

Lately I see more and more Delphi developers getting into debates with C# developers, and they are confronted with an attitude and belief-system that quite frankly is utter nonsense. It’s not based on history or facts but disputes rooted in childish “isms”. A mix of old habits and an unhealthy obsession with the notion that new means better.

It is nothing short of an intellectual emergency.

Old is bad?

If that is the case then C/C++ should be utter shit, because C is 3 years older than Pascal. Pascal was created to replace C and get away from (amongst other things) the absurd and time-consuming header file practice, not to mention the recursive hell that still occur when headers reference other headers and conditional symbols pile up.

Do you know how curly brackets became popular? Jump into your nearest Tardis and set the destination to the mid 1960’s. Back when mainframes were the size of Portugal and 1024 bytes of memory was the bomb. You see, back then memory was sparse and pretty much all the languages (or toolkits) needed to save space. So the { } curly’s were used as tokens for words like “begin” and “end”. Why? Because one ascii byte is less than five ascii bytes. Every byte counts when you only have a few Kb of memory.

Ps: when I use the word toolkit here I really mean a random soup of asm snippets on a loom. The bones collected to make C was actually a mix of homebrew toolkits made by different engineering teams at the time. And each of these toolkits had their own fanclub /slash/ legal representation; because all of them, miraculously, had invented the exact same thing at the same time before everyone else. These guys kept on fighting over who owned what well into the Watcom era. It’s probably the longest standing demonstration of a grown-up tantrum since the roman emperor caligula outlawed clothing.

Fast forward to 1970 and the amount of memory on mainframes had already doubled a few times just like Moore had predicted five years earlier. The C standard that Dennis Ritchie had painstakingly wrangled from the cold, greedy hands of grumpy old technicians was due for an upgrade. Tests were done by very dull, very serious men in grey clothing – and in their horror they discovered that human beings would interact with words and phrases faster than symbols and glyphs. So a word like “begin” would be picked up by the brain faster than {. It is just knockout stuff isn’t it.

You cannot praise Anders as a genius architect and at the same time throw his life work in the thrash. Perhaps you should ask yourself why Anders loved object pascal so much that he dedicated half his career making sure universities had access to it.

Living on the edge with a red tie, Nicolaus Wirth rocks the scene!

Living on the edge, Nicolaus Wirth rocks the scene!

Enter Niklaus Wirth, a man so clever and academic that he doesn’t catch exceptions; He is the exception. But next to his colleagues over at Berkley our dear Niklaus is the proverbial rock-star. He decided that in order to make programming more accessible for students, easier to learn and above all, establishing a standard for safer code, that they should take science to heart and include that in a programming language design. So he put on his gravest lab coat and began to implement “a better C” where he applied the research that words are picked up by the brain faster than tokens and symbols. Nine months later pascal was born, and to celebrate Niklaus used colored socks that day. A risky bit of business that could have cost him his seat at the strangely brown sorority club, in which he was master of ceremonies. But hey, Nikolaus is a rebel and this was the swinging 70’s after all.

My point? This is why pascal uses “begin” and “end” as opposed to { and }. It’s one of the many aspects of the Pascal language that makes it so easy to master.

Whimsical satire aside: with the knowledge that C is actually older than pascal, not to mention that it is identical in features, depth and complexity to Pascal– what is it going to be? Do you still cling to the notion that older have to be of less value than new? Or could it in fact be that you haven’t really looked closer at object pascal to begin with? Have you even installed a modern version of Delphi and given it a worthy test drive? Or do you just speak out based on what you like and know, as opposed to what you don’t know and haven’t even tried.

My 8-year-old daughter is like that. She makes up her mind about food before she has even tasted it. Surely a trained engineer can do better?

And I say that because — if we follow your line of reasoning, C/C++ should be incapable of producing C#. How can something old and outdated possibly produce something new and exciting? You do know that C# is written in C and assembler right? And that Java and all the other high level languages out there didn’t spontaneously self assemble. They are all written in C, assembler or pascal. And there is a reason for that.

Just like nature have different levels, from particles to minerals, from minerals to bacteria, from bacteria to plants, from plants to insects (and so on) — so does a computer. Each of these levels have laws and principles that govern them, and no matter how much you want to introduce something from a higher level into a lower level – that’s just not how reality works.

foodchain

The fact that I have to explain this in 2016 demonstrates how education and academia has degraded after C and Pascal was replaced with Java.

The law is very simple: each level can only emit complexity upwards. So assembler is used to produce C and pascal, C and pascal is used to produce C++ and object pascal, C++ and object pascal is used to bring about every other single piece of tech you take for granted.

So when you criticize Delphi but use C#, you are just underlying that you dont really understand the difference between archetypical languages and context based languages.

It’s like saying Visual Basic 5 is cooler than assembler. You don’t debate with people like that, you just look at them with pity. I don’t know how many years a software engineering  degree is elsewhere, but if you just spent 3 years in college and 2 years at a university followed by one or two additional years getting your diploma – and this is the level of insight you graduate with then I pity you. And I pity your employer because sooner or later you will bring about the apocalypse.

If we apply rational logical thought to the proposition at hand, we cannot but conclude that the notion of “new is always better” is false. It is to mistake marketing for facts and capitalism for science. The fundamental principles of computing wont change because you favour a high-level programming language. The C/C++ embedded market wont magically roll over, because C# is a mitigated disaster on embedded devices. And I speak from experience, not preference.

Your teacher should have taught you this: “In a computer, like nature, the past is always alive“. Look closely at your brand new PC or Mac: Bios, Vesa screen modes, non linear memory, hardware interrupt vectors and a cpu that supports no less than 3 instruction sets. You have layer on layer of increasing complexity (a.k.a “the past is still there”). Why not call up Linus Torvalds and ask him why he’s not using C# in his line of work; or NVidia why their GPU pipeline only ships with C headers and not C# classes. I sure as hell wouldn’t want to be on the receiving end of that call.

The only C# compiler worth using, is actually RemObjects C# (the Elements compiler). Because unlike mono or ms’s compilers, Elements builds proper binaries on the same level as C/C++ and Pascal. You can actually write kernel modules with Elements if you like, something you cant do with Mono or Visual Studio. Heck, .net services still require a service host on Windows.

C# is better than Delphi?

Anders Hejlsberg, the father of many languages

Anders Hejlsberg, the father of many languages

Actually, it’s not. What today is known as .net, from its CIL intermediate language right down to the global assembly cache is pretty much Delphi with a new syntax parser on top. Yes you read that correctly the entire .net product family is practically Delphi refactored. The whole bytecode regime was one of the last skunkwork projects Anders worked on, and was actually described on Borland’s news-servers years before it magically re-appeared as Microsoft’s flagship product.

Anders Hejlsberg created and prototyped these technologies (as did many other companies, since Java made bytecodes cool) while he was working at Borland. And indeed, the father of C# and entire dot net stack is none other than the father of Delphi.

At Borland (in the borland.language newsfeed I vividly remember) these technologies went mentioned under the codename “portable Delphi”. I followed the thread on the news servers for quite some time. It was no secret that Borland was tinkering away on the next big thing, and I believe it was ultimately Borland’s response to Java. Because at the time Java had eaten quite a chunk of Borlands educational seat licenses. And had Anders continued working for Borland, Delphi and C++ builder would perhaps be bytecode based today.

Factoid: The first project Anders was assigned to when he joined Microsoft, was J#. Microsoft’s attempt at hijacking the Java language. Microsoft lost the legal battle and was forced to pivot. The end result would be C#.

So when you, the C# fanboy, propose that Delphi is somehow inferior to dot net, or that object pascal is outdated and technically behind C# you quite frankly have no clue what you are talking about. Is it generics? We have that. Is a rich and powerful RTTI? Got that covered. Anonymous procedures? Its there too. In fact, the whole misunderstanding here stems from the fact that C# developers generally think “Delphi 7” is what Delphi is all about. Which is the same that saying that .Net  is what came out in version 1. Who the hell uses Delphi 7 in our day and age.

But just to make this crystal clear: Anders Hejlsberg is not just the father of C# and dot net: he is also the father of Delphi. And before that he gave birth to Turbo Pascal. In fact, Anders have 3 (if not four) Pascal development systems behind him before he came to Microsoft.

Borland, land of the free

Borland, the company Anders worked for, was put out to pasture by Microsoft. Bill Gates launched a financial onslaught that few companies on the planet could have endured. The reason was that Microsoft really had the worst developer tools on the market (Visual Basic and Visual C++), while Borland represented the very best. So for a long time Borland utterly demolished Microsoft when it came to software development.

When Borland later began to flirt with Linux (Delphi Kylix) it must have made the R&D department at Microsoft piss themselves. Delphi is responsible for thousands of desktop applications synonymous with Windows – and if Linux users got their hands on RAD tools like Delphi, with thousands of applications ready to be ported over? Do the math. You have to remember that this was before cloud computing. Microsoft didn’t have a fallback strategy and depended on desktop and server sales. Linux could do everything Windows could, but it lacked the desktop applications, the user friendliness and features Borland made simple.

Weird Science, a fun but completely fictional movie from 1985. I just posted this to liven up an otherwise boring trip down memory lane

Wishing for stuff doesn’t make it true, except in the movies

What Microsoft did was, basically, to buy out Anders (they also picked up a huge part of Borlands R&D department) from Borland and at the same time attack the company from every angle. Drowning them in bullshit lawsuits, patent claims and all the nasty, cowardly practices Microsoft was into back then (we all remember Netscape and how that went).

Bill Gates called Anders up personally and gave him “an offer I could not refuse“. Whatever that means.

So to put this myth to bed and be done with it: C# as a language and dot net as a technology did not originate with Microsoft. It has Borland written all over it and its architecture is intimately joined at the hip with Delphi and C++ builder. Companies like Microsoft dont innovate any more, they buy competence and re-brands it as their own.

Dot Net framework, you mean the VCL?

If that is not enough, take a long, hard look at how the .net framework is organized. Pay special attention to the features and methods of the base-classes and RTTI. What you are looking at is the VCL, the visual component library, Delphi and C++ builder’s run time library.

If you know Delphi and it’s RTL intimately as well as C#, you quickly recognize the architecture. The syntax might be different, but the organization and names are all too familiar.

There was never any secret where this came from, but it’s not advertised either so I don’t blame people for not recognizing it. But when people start to behave nasty over what should be either a passionate hobby or a professional discipline, then truth must be told. The very foundation of the dot net framework is taken straight out of the VCL.

And why shouldn’t it? The whole platform was whispered about under the codename “portable Delphi” to begin with, it was even designed by the same author – so why should it come as a surprise that it’s basically Delphi reincarnated with a different syntax parser? Either way a few bells should start ringing in your head – telling you that you may have been wrong about Delphi and object pascal in general. Probably because you know very little about Delphi to begin with.

So, what you perhaps believe sets C# apart from Delphi, what you imagine gives dot net an advantage over Delphi – is in reality the opposite: it is Delphi. It originated in Delphi, was inspired by Delphi and flows straight from the inventor of Delphi himself. Anders is responsible for two distinct object pascal development platforms prior to working for Microsoft. Take some time to reflect on that.

You cannot praise Anders as a genius architect and at the same time throw his life work in the thrash. Perhaps you should ask yourself why Anders loved object pascal so much that he dedicated half his career making sure universities had access to it.

Strength in diversity

The fact that C/C++ is alive and well today serves to prove my point: age has no meaning when dealing with fundamental technology. These languages embody the fundamental principles of computing. They don’t grow old because they represent a fundamental level that can never be replaced. There is now law that says C must be there, what persists is ultimately the features that C brings – spanning from assembly to the desktop. Those same features can be found only in one other language, namely Pascal.

Things are the way they are for a reason. And programmers before you established standards that address things you might not have to deal with, but that’s because they dealt with it before you. Just because you don’t see the plumbing doesn’t mean it’s not there.

These are the compilers responsible for the foundation on which all other languages, operating-systems, applications and services rest. Remove those, and the whole house of cards come crashing down.

So C/C++ and object pascal continue to exist, thrive even, precisely because they interface with the hardware itself, not a virtual machine or bytecode format. Delphi as a product is naturally tailored for operating-systems, as is C/C++. But that is because we use programming languages to create applications. There is nothing in the way of someone using compilers like freepascal and writing an operative system from scratch.

Ultibo is an ARM operative system written purely in object pascal

Ultibo is an ARM operative system written purely in object pascal

By contrast, C#, Java and Visual Basic are context dependent languages. They exist purely in context with an operative system or run-time environment. They can’t exist without these and lack the depth C and pascal brings to the table. The depth to span many levels, from the lowest all the way up to cloud and domain driven programming.

Just look at Android, an operative system written in Java right? Well that’s not really true. The Android operative system depends on a native bootstrap, a piece of code that establishes the run-time environment for Java applications. So when you fire up your Android phone, tablet or set-top box, the first thing to boot is a loading mechanism written in vanilla C. This in turn mounts a filesystem and continues to loads in bunch of native drivers. Drivers for the display, storage, wi-fi and all the hardware components that ultimately make up your SoC (system on a chip).

Its only after this long process that the environment is capable of hosting the Java virtual machine. Then and only then is the device capable of executing Java. This is why languages like Java is called context-based, because they run in context with an established system.

7709-f-1

Notice the “C” part of the Java NDK? Wasnt that exactly what Java was supposed to solve? All that hype and here you are, back at where you started!

So Java, which for the past decade or more have bombarded us with advertising like “platform independence” is a total flop! Its like communism and looks good on paper, but once you start testing these ideas in real life – they just don’t work.

Turned out that the java virtual machine when running on small devices like phones and tablets, choked the CPU. Which in turn used more electricity to keep up, which in turn rendered the battery life of these devices null and void. Talk about irony. All that hype and Java couldn’t even deliver its most basic promise: platform independence. And you removed C and pascal from our universities for this? It’s a disaster en-par with the burning of the great library of Alexandria.

There is more than a drop of ego and arrogance in this material. Young people especially like to believe they are smarter than those who lived before them. It’s a very selfish idea that really need to be addressed. Take someone like Alan Turing for example, the inventor of what we call computers today. This man assembled and created the first computer by hand. He even hand carved the wheels that made the mechanism run (which was mechanical during the second world war). I would happily bet $1000 that Alan could out program the lot of us had he gotten the chance. He has been dead for many years, but to imagine yourself smarter or more capable than him would be the definition of arrogance.

PS: This is where we come into the realm of wisdom versus knowledge. The difference may be subtle, but it’s so important to distinguish between them.

C# would have been a sad little language without the work of this man, Miguel de icaza

C# would have been a sad little language without the work of this man, Miguel de icaza

Either way without the native context to support it Java turned out to be a blundering behemoth. And its the exact same with C#. This is why Miguel de Icaza work on Mono has been so instrumental to the success of dot net and C# in particular. It was Miguel and his team that pushed C# into native compilation versus JIT for mobile devices. C# would never have been allowed on IOS or Android otherwise.

So that is the difference between archetypical languages and context based languages. Hopefully you now see why this negative attitude towards Delphi and object pascal doesn’t make sense. It really is just as stupid as these cults that believe in dinosaurs and that human beings once lived side by side with a T-rex. Anyone with a highschool diploma should instinctively know better.

The problem is when people in great enough numbers start to believe in this nonsense, because then they can actually damage important mechanisms of our eco-system. Insane beliefs really is dangerous both socially and technically.

Grow up!

I have no problem using C# and consider it a valuable tool in my toolbox. Same with Delphi. But when it comes to kick-ass desktop applications for Windows and OS X, Delphi has no rival. It gives you a productive edge that I havent found in any other language. And I don’t base that on preference. I base that on engineering and the years of development invested in Delphi by both Borland and Embarcadero.

I just find it odd that C# developers seem to lack the ability to speak well about Delphi, because it’s a tell-tell sign of immaturity to be incapable of that. Object pascal and C++ is identical in every way (except multiple inheritance, which even C++ programmers shun like the plague so no loss there). Why then should educated programmers find it reasonable to praise C but thrash pascal? It’s like saying that one car is faster than another – but when you pop the hood they have the same engine. The exact same machine code generator even.  It is psychological masturbation. We are coders, we shouldnt do things like that. We should examine the system carefully and then make up our minds.

And what is a programming language anyway? It is the ability to describe the solution to a problem. The ability to define and control how a series of activities execute. Each language represents a methodology for solving problems. A mindset. Nothing more.

Languages like C, C++ and object pascal will always exist, they will always thrive, because they are instrumental to everything else (desktop, operative system, services, everything that makes a computer capable of what it does). It doesn’t mean that Delphi is the solution to all problems, far from it. And that is ok.

Context based languages are more often than not designed for a purpose. Java was initially designed as a network language. This makes Java infinitely more suitable for dealing with that line of work. You can do the exact same in C but you would have to add a few libraries and establish a platform. Which is exactly what C and pascal is good at, implementing fundamental technology. Object pascal is a language you would use when creating other languages. C is more popular for virtual machine development, no doubt there, but there is nothing in C/C++ that you wont find in object pascal.

Interestingly, languages that many believe is long since extinct, like Cobol, are actually alive. Cobol was created to deal with large stacks of numbers and to push associated data through formulas. That is its context and no other language is better suited for the banking sector than Cobol. Which incidentally is the one place where Cobol is used to this day. And why not? Why waste millions trying to hammer in a nail with a hacksaw? Why not use a language especially designed for that purpose? SQL is likewise a language especially tailored for its tasks. I don’t hear anyone bitching about how old and outdated that is. It wont grow old because it embodies the principle of relational data.

And just what exactly is old about Delphi? Generics? Posix support? Weak and strong references? Garbage collection? Attributes? class, record and enum helpers? Dependency injection? Model view controller frameworks? The ability to write assembler side by side with pascal? The ability to link with C/C++ objects? Cross compilation for Windows, OS X and mobile platforms? What manner of logic are you using where you praise other languages for these features, but for Delphi the same features makes it old? Stop repeating what someone once told you like a mindless parrot and investigate. You are an engineer, a field of science; start acting like one.

In my work I use several languages: JavaScript, Delphi, Smart Pascal, Mono C# and even Python. I have also fallen in love with LUA believe it or not.

For web-pages I prefer Smart pascal which compiles to JavaScript (node.js is fantastic). Then you have shell scripting, a touch of x86 assembly code, a dash of bash on my Linux box and .. well, there is strength in diversity.

It all depends on what is best for the customer and what language yields the most robust solution. Sometimes C# is better suited for my clients infrastructure, other times Delphi makes more sense. But if I was to pick a single language to cover all of them, it would either be object pascal or C++ (gcc). These really have no rivals and will never go away.

These are the compilers responsible for the foundation on which all other languages, operating-systems, applications and services rest. Remove those, and the whole house of cards come crashing down.

So using many languages is actually an enriching experience. It brings more depth to our way of thinking and (consequently) affect the quality of our products.

Anyways, I believe congratulations are in order, because if your favorite language is C#, you have in fact been using Delphi without knowing it. There is so much Delphi in C# and they both originate from the same author. So why not pay a visit to C#’s older brother?

So go download Delphi, Lazarus and Remobjects Oxygene – enjoy the decades of research and excellence these dialects represent. It won’t hurt you, it will probably do you a great deal of good.

Smart Mobile Studio, datasets and websockets

September 25, 2015 2 comments

It's all about the bytes

It’s all about the bytes

If you have been playing with the beta you have probably noticed a few cool new units in the RTL. First there is system.memory, system.typecon, followed by system.streams. And last but not least: system.dataset.

What’s so special about these units? Well, they give you the same level of low-level (read: byte) access as you enjoy under Delphi. If you know your way around JavaScript you must have noticed that this is a sensitive subject, a really sensitive subject (!). JavaScript is hopeless when it comes to binary data, and there is no such thing as a “stream” in the browser.

You wont believe some of the solutions out there which pass for “top notch” JavaScript. Most of them involving cheap hacks like stuffing bin-hex’ed data into strings. This means you need a separate layer of encoding before you push the data to a server (or de-serialize it to JSON). Well we dont have that problem 🙂

So, with all these units firmly in place we can now enjoy the same level of power, simplicity and elegance as under native Delphi or freepascal. And this is where TW3Dataset comes in. Because it’s actually a fully OOP dataset which saves to a very simple binary format (!)

TW3Dataset

Fast, binary, pure object pascal

Fast, binary, pure object pascal

Let’s say you have a mobile solution which needs to store uniform data. It can be a phone-book, a list of high-scores for a game, time samples or your latest RSS feed poll. Traditionally you would sculpt it all using arrays and then serialize the whole shabam to JSON; or even worse: make use of WebSQL, PouchDB, TaffyDB or other JavaScript dependencies.

Well that is now a thing of the past. For simple in-memory dataset chores, TW3Dataset covers your basic needs more than adequate. It also saves to TStream, meaning that saving and loading from the new storage system (more about that later) is reduced to a “one liner”.

Is it hard to use?

Nope. In fact it’s even simpler to use than TClientDataset under Delphi (or more or less identical, except with no indexing just yet). Here is a short example of how to create a dataset, populate it and save it to a stream:

var
  mDataset: TW3Dataset;
  mData:  TStream;
  x:  Integer;
begin
  mDataset := TW3Dataset.create;
  try

    mDataset.FieldDefs.Add('id',ftAutoInc);
    mDataset.FieldDefs.add('name',ftString);
    mDataset.CreateDataset;

    for x:=1 to 1000 do
    begin
      mDataset.append;
      mDataset.Fields.FieldByName('name').AsString := 'Name #' + x.toString;
      mDataset.Post;
    end;

    mData := TMemoryStream.create;
    try
      mDataset.SaveToStream(mData);

      writeln('Dataset bytesize = ' + mData.size.toString);

    finally
      mData.free;
    end;

  finally
    mDataset.free;
  end;

As you can see from the code above some calculated field types are supported (Autoinc), and TW3Dataset is going to see some heavy expansion as it’s merged with my universal DB framework. Think FireDac but for the browser.

Other cool stuff

If TW3Dataset is to limited and you want SQL support, Smart Mobile Studio now ships with SQLite compiled for JavaScript. Yes you read right, SQlite has been compiled for JavaScript and reads, writes and generates identical files to the native SQLite versions. That’s pretty darn cool (although it does add 1 megabyte to your apps).

So in fact, you can now create and populate a full SQLite DB and just ship it to your server “as is”. Add encryption and you have a pretty good syncronization layer to play with.

W3C, WC3 or just WC

SQLite used to be built into modern browsers, then dubbed “WebSQL”; but in their infinite wisdom the W3C decided to deprecate that and go for a key-value pair storage system reminicient of MongoDB on wellfare. The good news is that our RTL is now so powerful that we dont need to care about the whims of the W3C any more. Just like Delphi revolutionized database driven desktop application development in the 90’s — we will revolutionize DB driven JavaScript applications for the modern age.

True binary streams? You got it! Compression routines ported directly from Delphi? No problem. In-memory databases? Yup!

Websockets made human

Websocket, oh what a magical buzzword! With all the hype you should think it was black magic. Well our SmartCL.Inet unit has now been expanded with a very cool WebSocket client. This means that if you want to talk directly with a nodeJS installation, or perhaps even better — write your own websocket server and get into Amazon cloud services for next to nothing, the RTL now makes that possible.

Writing a websocket client is now reduced to:

  FSocket.Connect("wss://echo.websocket.org",[]);
  FSocket.OnOpen:=Procedure (Sender:TW3WebSocket)
    begin
      writeln("Socket connected");
      Sender.Write("Rock it with HTML5 WebSocket");
    end;
  FSocket.OnClosed:=procedure (Sender:TW3WebSocket)
    begin
      Writeln("Socket disconnected");
    end;
  FSocket.OnMessage:=Procedure (Sender:TW3WebSocket;Data:String)
    begin
      Writeln("Data received:" + Data);
    end;
  FSocket.OnError:=Procedure (Sender:TW3WebSocket)
    begin
      writeln("Socket has errors");
    end;

File storage for the rest of us

Sandboxed files and JavaScript is a complete mess. JavaScript is utterly event driven, wich means that everything requires callback handlers. From accessing a medium, to being allowed said access to creating a writer and finally do the actual writing (phew!). And each event handler must denote the calling context just for the hell of it.

To make this usable by human beings, the RTL introduces a concept called “file-actions”. They work more or less like TAction under Delphi, but are especially tailored for loading and saving data in a uniform way.

Here is an example of saving and then loading back a small file:

procedure TForm1.W3Button4Click(Sender: TObject);
var
  mAction:  TFileActivitySave;
begin
  mAction:=TFileActivitySave.Create;
  mAction.FileName:="document.txt";
  mAction.FileMode:=fmPermanent;
  mAction.OnExecute:=procedure (sender:Tobject)
    begin
      writeln("Saving done");
      var mLoader:=TFileActivityLoad.Create;
      mLoader.FileData:=TMemoryStream.Create;
      mLoader.FileName:='document.txt';
      mLoader.FileMode:=fmPermanent;
      mLoader.OnExecute:=Procedure (sender:TObject)
        begin
          writeln("Loading done!");
          writeln("Bytes ready = " + mLoader.FileData.Size.tostring);
          try
            var mReader:=TReader.Create(mLoader.FileData);
            writeln(mReader.ReadString);
          except
            on e: exception do
            writeln(e.message);
          end;
        end;
      mLoader.Execute;
    end;
  mAction.FileData:=TMemoryStream.Create;
  var mTemp:=TWriter.Create(mAction.FileData);
  mTemp.WriteString("This is some data");
  mAction.Execute;
end;

As you begin to explore the additions to the RTL in the next release, you will find that a lot of time and effort has gone into this. So I hope everyone get’s excited when they what they can now do — and that object pascal is the perfect language for HTML5 and mobile application development!

Hire a Delphi superhero!

May 18, 2015 2 comments

This is your chance to hire a Delphi super-hero, namely me! That’s right, I’m presently in the market for a new full-time job!

Note: if you need people for a short project then I personally must decline, but I can forward your position at Delphi Developer (4.000 Delphi developers) and here on my website (average of 15.000 visitors per month)

I must underline the obvious, namely that I live in Norway with no intention of moving. However, I have a fully established office which has been used for three consecutive years. So I am used to working remotely, communicating through Skype, Messenger, Google Meetup’s and similar utilities. This has worked brilliantly so far. Full time employment over the internet is finally a reality.

For Norwegian companies I can offer 50% commuting (100% in Vestfold) from Oslo to Akershus in the north, to Sandefjord and Larvik in the south. The reason I can only offer 50% is because I have custody for two children every other week, which means I am unable to drive long distances in combination with school and similar factors.

You need an awesome Delphi developer? Then I'm you man!

Jon Lennart Aasenden, 41 years old senior software engineer using Delphi

My experience is too long to list here, but if you are in the market for a dedicated software engineer then you have come to the right place. Simply drop me a line on e-mail (address at the end of this post) and I will get back to you with a full resume ASAP.

In short: I have worked with Delphi for 15 years, I am the author of Smart Mobile Studio and wrote the VJL (run-time library, similar to VCL) from scratch. I have worked for some of the biggest companies in Norway. I have good papers, excellent reputation and is recognized for my skill, positive attitude and “yes we can” mentality.

Before my 15 years of Delphi I spent roughly the same amount of time programming games, multimedia and contributing to the demo scene. In my view, demo programming is one of the most rewarding intellectual exercises you can engage in. It teaches you how to think and prepares you for “real life solutions” as a professional.

In my career as a Delphi developer I have written several commercial products; ranging from a complete invoice and credit management application, games and multimedia products, serial management components for Delphi, backup clients and much, much more.

No matter what your company works with, as long as it’s Delphi I’m the right man for the job.

My kind of job

It goes without saying that I am looking for a purely Delphi centric position. Although I have worked with C++ and C# and have no problems navigating and adding to such a codebase — there can be no doubt that Delphi is the environment where I excel at my work.

I bring with me a solid insight into HTML5/JavaScript and am willing to teach and educate your existing staff in Smart Mobile Studio, which I am also the founder and inventor of; bringing your team up to speed with the next step in computing: namely distributed cloud computing. At which JavaScript and Object Pascal play central roles.

Torro Invoice

Torro Invoice

So if you are moving towards cloud computing or want to leverage the cloud for service oriented architectures using RPC or REST servers, I am the guy you want to talk to. Especially if you want to use object pascal and leverage nodeJS, JavaScript and the whole spectrum of mobile devices. In short, with me on your team you will have a huge advantage.

What you get is 15 years of Delphi experience; and before that, 15 years of software authoring, demo and game development on 16 bit machines.

Good qualities

I have included a small list of “ad-hoc” qualifications. The list below does not serve as anything other than indicators. I have been a professional programmer for many, many years and to list every single piece of technology is pointless. But hopefully the list is enough to give you an idea of my qualifications, depth and diversity.

  1. Write custom Delphi controls and UI widgets
  2. Create control packages and resource management for our projects
  3. Optimize procedures by hand by refactoring variables, parameters and datatypes.
  4. Write a database engine from scratch and also use existing engines such as Elevatedb, MSSQL, Oracle, Firebird, SQLite and MySQL
  5. Work with Windows API directly
  6. Work with freepascal and port code to Linux and Macintosh
  7. Write networking libraries, custom protocols and high-speed multithreaded servers
  8. Use classical frameworks like Indy, Internet Direct and other out of box solutions
  9. Write advanced parsers for source-code and any complex text-format
  10. Write a real-life compiler and IDE (www.smartmobilestudio.com and quartexpascal.wordpress.com)
  11. Create a graphics library from scratch, supporting different pixel formats
  12. Work with large files, performing complex operations on gigabytes and terabytes
  13. Write mobile and desktop applications using Smart Mobile Studio and FireMonkey
  14. Create your server environment, architect your RPC (remote procedure call) service infrastructure
  15. Implement custom services through nodeJS, Smart Mobile Studio, Delphi and FreePascal
  16. Use Remobjects product line: such as RemObjects SDK, Hydra and Data-abstract
  17. Link with C and C++ object files
  18. Use project management tools, scrum, SVN, github and package management
  19. Write, export and import DLL files; both written in and imported to Delphi
  20. Write solid architectures using moderns technologies: xsl schemas, stylesheet validation, rest, node services

.. and much, much more!

Note: While you may not have a need for all of these qualifications (or any one of them), the understanding which the listed topics demand that I master is of great value to you as an employer. The more knowledge and hands-on experience a programmer can offer, the better qualified he is to deal with real-life problems.

Tools of the trade

All programmers have a toolbox with frameworks, code collections or third party components they use. These are the products I always try to use in projects. I have a huge library of components I have bought over the years, and I have no problem getting to know whatever toolkit you prefer. Below is a short list of my favorite tools, things I always carry with me to work.

  • RemObjects SDK to isolate business logic in RPC servers, this allows us to use the same back-end for both desktop, mobile and html5 javascript.
  • RemObject Data Abstract
  • Remobjects Hydra
  • ElevateDB and DBIsam (DBIsam for Delphi 7-2010, ElevateDB for the rest)
  • Developer Express component suite
  • TMS Aurelius database framework
  • MyDac – MySQL data access components
  • MSDac – Microsoft SQL server data access components
  • Raize component suite
  • FreePascal and Lazarus for OS X and Linux services
  • Mono/C# when customers demands it (or Visual Studio)
  • Firemonkey when that is in demand
  • Smart Mobile Studio for mobile applications, cloud services [nodejs] and HTML5/Web

Tools and utilities

  • Beyond compare (compare versions of the same file)
  • Total commander (excellent search inside files in a directory, recursive)
  • DBGen – A program written by me for turning Interbase/firebird databases into classes. Essentially my twist on Aurelius from TMS, except here you generate fixed classes in XML Data Mapping style.

Note: For large projects which needs customization I tend to write tools and converters myself. The DBgen tool was created when I worked at a company which used Firebird. They had two people who hand wrote wrapper classes for nearly 100 tables (!) They had used almost two months and covered about half. I wrote the generator in two days and it generates perfect wrappers for all tables in less than 10 minutes.

Safety first

As a father of three I am only interested in stable, long-term employments. Quick projects (unless we are talking six figures) are of no interest, nor do I want to work for startup groups or kickstarters. I am quite frankly not in a position to take risks; children comes first.

many of my products are available for purchase online

Many of my products are available for purchase online

I am used to working from my own office. I was initially skeptical to that, but it has turned out to work brilliantly. I am always available (within working hours Norwegian time) on Skype, Google, Messenger and phone.

Languages work format

For the past year I have been working full time with C# under Microsoft Visual Studio. Before that I used Mono for about four months, which I must admit I prefer over Visual Studio, and wrote a few iPhone and iPad apps. So I am not a complete stranger to C# or other popular languages. But as I have underlined in my posts on countless occations – object pascal is a better language. It is truly sad that Borland was brought down all those years ago, because with them they dragged what must be one of the most profound and influential languages of our time.

That said, I have no scruples about mixing and matching languages. Be it Remobjects Oxygene, C#, C++ or Object Pascal. I also do some mocking in BlitzBasic which is without question the fastest (execution wise) of the bunch, but not really suitable for enterprise level applications.

The point here being that I am versatile, prepared to learn new things and not afraid to ask. It would be a miracle if a man could walk through life without learning something new, so it’s important to show respect and be remain humble. Different languages represents different approaches to problem solving, and that is an important aspect of what it means to be a programmer.

The only thing I ask is that I am properly informed regarding the language you expect me to deliver in – and that I receive the required training or courseware in due time. Do not expect super-hero status in C#, I am absolutely above average, but it cannot be compared to my efficiency and excellence using Delphi.

Contact

If you are interested in hiring me for your company, then don’t hesitate to contact me for a resume. You may e-mail me at lennart.aasenden AT gmail dot com.

Angular? Nothing new under the sun

May 14, 2015 3 comments

In a previous post I sort of slaughtered Angular.js and Bootstrap. Partly because these libraries represent “a prototypical solution to the lack of classes, inheritance and a VMT”. My point was simply that if you sat down to program Angular in Smart Pascal or Delphi, you would use ordinary inheritance to solve many of the more tricky subjects angular imposes on us.

And in so doing, no one would bat an eyelid – nor would it be hyped to the point where people believe they can cure cancer with it. It would just be another MVC framework, one among thousands. But since Google made it and spontaneously throws a few workshops around it – people go nuts and imagine that it’s just so much better than anything else out there.

Ladies and gentlemen, that is just sad, disappointing and intellectually disturbing. As a result I no longer believe our species will ever reach the stars, quite simply because the wast majority of our human population are indistinguishable from cattle.

Apple made MVC hot again, but it's an old re-hash from yonder days

Apple made MVC hot again, but it’s really a re-hash from ‘yonder days

Now the blogging I do here can only be one or the other: either a short post (like this one) or a long, more detailed post. It depends on how much time I have and how interesting the topic is. But it’s rare that I post a medium-sized article. I mention this because I have gotten feedback like “Oh you have completely misunderstood MVC” — which is simply not true. In fact I work with angular on a daily basis at work.

But that doesn’t mean I will publish a dissertation on the subject just to underline that I do have experience with MVC. Readers of my blog will know that I’m not a proud individual. If I have done wrong, said wrong or coded wrong I have no problem saying I’m sorry. It is after all impossible to go through life without making mistakes. So if I came across as just shooting things because “I don’t like them” mentality, that was not what I meant; What I meant is that I don’t like them because-of a wide range of valid reasons.

That’s why I cut the MVC criticism short because (quite frankly) I thought my issues would be common knowledge. Clearly they are not so I will try to bring more light to the subject in this post.

MVC is hardly new tech

MVC is not new. It’s bloody ancient and it tends to arise and appear towards the height of a platform’s popularity. So if nothing else it’s a clear indicator that right now JavaScript and HTML5 is at its height, and in the years to come we will see a steady decline in the way we use this technology. The timeframe coincides with the emergence of the next level in computing: namely the cloud and it’s potential.

So for those that believe MVC is something new and fresh – compared to Delphi which is termed old and useless, think again. First of all the oldest programming language still alive in the world is C, which is around six years older than pascal (just to kill that misconception if you suffer from it). MVC is not as old as that, but it’s pretty damn vintage! I remember doing MVC coding on the 16-bit Amiga computer some 25 years ago.

A huge selection of the OS can be automated through AREXX

A huge selection of the OS can be automated through AREXX

One of the lesser known aspects of Amiga OS today is the fact that it came with a built-in automation system called AREXX. It was absolutely brilliant. What programmers did was to define and register AREXX ports in their native applications. These ports worked a lot like socket’s do, in that you can send and receive messages on them. Once exported, which was done by calling the OS informing it about the functions you made publicly available, these functions could be automated and called from any AREXX script.

So what people did was to design the view using a MUI editor (magic user interface), write an application controller and essentially create existing OS level controllers and classes to create an application. Whenever you needed something special you would just import the functionality of a commercial program and use those functions to do your work.

  • Write application controller using AREXX
  • Design views using a MUI (magic user interface) designer
  • Create and maintain OS level objects and controllers
  • Maintain a persistent data model, similar to COM’s property-bag

Listing REXX exported controller ports under Amiga OS

Listing REXX exported controller ports under Amiga OS

Even seemingly new techniques like “MVB”, where you bind or link visual elements to a property in the model could be achieved. In fact on the Amiga you could create fully working, complex and advanced applications using nothing but AREXX scripting. So for those of you that believe the MVC pattern is new and fresh of the mint, sorry to burst your bubble but it’s most likely older than you.

Remember that myself and hundreds of thousands of others were using this more than a decade before Microsoft Windows was even conceived of. I remember bursting out in laughter when Apple introduced their “Automator” script system back in the day. Why? Because the Amiga has that 25 years ago! And it get’s better because in january (I think it was) Apple announced that OS X was now the pinnacle of operative systems, because now they supported REXX scripting (!)

You know what? For ages now people have blamed Microsoft for ripping off Apple’s ideas. But guess what, most of them didn’t come from Apple, but from a kick-ass 16 bit operative system and computer, 15 years ahead of the competition back in the early 80’s.

So Apple have (and there is no doubt here, they have even admitted it from time to time) robbed Commodore Amiga blind. And the fact that they are still lifting good ideas from a 30 year old operative system speaks volumes both about the Amiga and those out to duplicate its technology.

The bright side of MVC

We are all adults here and I take for granted that you have enough life experience to recognize that everything has dark side. It doesn’t matter what it is, there is always some side-effect or “potential for error” or risk assessment where an otherwise brilliant technology yields nothing but disaster. This is true for Delphi’s CCV (class, component, view) pattern, model view controller and dot net’s distributed assembly model. Take your pick, there is no universal technique which works everywhere without flaw, side-effect or zero potentiality for error.

Now the bright side of MVC is fairly easy to spot:

  • If architected with care, the same controller can be used on different views
  • Writing plugins or extending a MVC based framework can be fragmented
  • Under HTML/JS an application appears as a single document, but is in fact rendered by a route manager

If we have a look at these you will notice just like me, that the first one sort of comes naturally.  Right? I mean it’s not an “aha” moment you read here for the first time and just go “Oh my god I never thought about that before”.

The second point is likewise an immediate factoid that you locate if you spend 2-3 minutes thinking about how a 3 partite organized framework must look like. And the last point is cool but ultimately easy to achieve no matter what language, technique or pattern you subscribe to.

MVC applications quickly become a mess to work with

MVC applications quickly become a mess to work with, remember to be nazi at organizing

If I was to point out something positive about MVC, it would have to be the first point on the list, namely that if you carefully architect your controller – it can be used on multiple layout views. So a typical example of something “neat” is exactly that: when you have a website that supports both desktop and mobile devices.

The mobile layout will in the best of cases have the same elements but organized differently. Worst case scenario is when mobile and desktop views use different model fields, forcing your response controller to include logic which MVC was designed to avoid. But in general angular.js is brilliant at bringing this to the world of JavaScript, because it does make things easier for JavaScript developers.

Again, angular is just fantastic for programmers with no knowledge of true object-oriented programming. If all you know is JavaScript then odds are you wont be able to recognize “real” stuff. Your mind would simply exclude it as un-important because it’s alien to you.

Do a quick google on javascript 10kb demos and you will find hundreds of superb graphically demonstrations written in JavaScript. But I can tell you right now that nearly none of those demos are written by a programmer that only know and work with JavaScript. These are old coders like myself with a background in hacker groups, demo and intro programming – and they have been doing this for 30+ years.

The dark side of MVC

The downside of MVC, is that your views can (depending on the framework you use) have something like 30 to 100 controllers running at any given time. Apple is positively criminal when it comes to this. It’s common that a single form design, although a complex one, spawn’s as many as 100 controllers when pushed onto the display.

Why? Because a controller is not the same as “the form resize code”. Oh no, a controller can be anything from watching that you prefix text with a $ character, a routine which keeps track of an application state – to making sure a control horizontally stretch to match its neighbors. A controller is just a piece of code responsible for a single behavior; or better yet – a method which ensures a distinct type of behavior. It all depends on subject, object and toolkit.

Incidentally, this is why Java and C# programmers are so hopelessly dependent on garbage collection. Can you imagine working on a huge MVC iPad project where every single view has between 30 to 100 controllers? This means that every element has controllers taking care of scaling, rotation, movement, fading, projection, touch, routes and cross-view event delegation (phew!). And it’s recursive to you know, hosted user-controls are likewise MVC based! Not exactly an eco friendly, resource moderate technology is it? Not to mention the waste of CPU power just because of a damn coding practice.

Now apply that mess to a single threaded event based runtime engine and try catching an exception induced through an anonymous timer. That’s when the regret of chosing angular over object pascal hit’s you like a ton of bricks and you whole heartedly regret ditching a rock solid language and it’s super effective CCV pattern.

The human factor

By now you probably get that my problem with angular, jquery, bootstrap and it’s likes is not superficial. It’s not based on a whim or defined by something stupid like fanboy-ism. Quite the opposite. In my line of work you have to know several programming languages if you expect to survive. My favorite language is object pascal, everyone knows that, but all languages and techniques, patterns and practices have strength and weaknesses. And it’s your profession to pick the right tool for a job.

So for me personally as well as professionally, a programming language is just a vehicle for my experience. You can’t expect to use the same tool to fix every single problem. Although I must admit that Delphi and Smart Mobile Studio combined is akin to a proverbial swiss army sword.

Bigger the projects, wider the toolbox

Bigger the projects, wider the toolbox

So angular is fantastic if all you know is javascript. But if you come from a more classical background, like Delphi or C++ then angular will soon get on your nerves. Because after a little while you realize that most of the blubber posing as high-tech stocking fillers – exists primarily due to the lack of OOP. It represents the author’s means to overcome, bypass and make up for the fact that javascript is prototypical, not object oriented.

MVC delivers no true benefit over ordinary object pascal’s CCV (or C++ builder, or even C# for that matter). In Delphi everything angular represents (and jquery + bootstrap) would be solved far more elegantly using nothing but vanilla inheritance.

Take something simple, like using the same controller on two different views:

type

TMyView = Class(TCustomForm)
public
  procedure SomeControlOperation;virtual;abstract;
  procedure SomeOtherControlOperation;virtual;abstract;
end;

// Our mobile "view" takes care of the mobile displays
TMyViewMobile = Class(TMyView)
public
  //Layout and operations for mobile views
  procedure SomeControlOperation;override;
  procedure SomeOtherControlOperation;override;
end;

// Our desktop "view" takes care of larger displays
TMyViewDesktop = class(TMyView)
public
  //Layout and operations implemented for desktop displays
  procedure SomeControlOperation;override;
  procedure SomeOtherControlOperation;override;
end;

In the above example we isolate our API, which essentially act as our controller, as purely abstract methods in our ancestor class. We then derive and derive two separate variations, one for mobile and one for desktop. Each of these implement their kind of layout according to the rules we impose in our API (the public procedures).
Since this example only have two elements to support (desktop and mobile) it would be better to implement the desktop behavior directly in the root-ancestor (TMyView) and have TMyViewMobile override that for mobile devices. If nothing else than to keep our code tidy and neat.

Not a controller in sight, but easy to use, deploy and debug

Not a controller in sight, but easy to use, deploy and debug

What I have objected to about angular (et al) is this popular but certainly false notion that MVC produces better, faster, easier or cleaner results than the above. In fact the result would be exactly the same. I would also go so far as to say that using inheritance is cleaner and provides a human bonus: namely better organization. We don’t have to use X number of folders to organize views, controllers and model schemas – because they all make up 3 parts of a common whole. Why write 3 times as much code for absolutely no reason other than pretending to be cool?

So this popular notion that object pascal is old, ancient or past it’s expiration date (comparing to MVC) is simply not true. The entire misconception is based on ignorance and laziness, plain and simple. And how can I propose that? Because there is no technical benefit of choosing MVC over a traditional CCV object orientation. People chose the hyped re-hash because it’s easy, not because it’s technically excellent, faster, yields better results or any of those superlatives.

Just remember: with great simplicity emerges an even greater lack of diversity!

Aros, winner of most interesting OS

May 12, 2015 1 comment

Right. As most readers know by now – once a year I get this unstoppable desire to check out all things alternative in the realm of operative systems. I mean, it’s important to stay on top of our game as developers, and the only way to do that is to make an effort. In this case an effort to look at upcoming systems which may become our own sometime in the future.

And as always one of the criteria is (drumroll) that they should support Object Pascal.

The AROS operative system

Aros desktop, not exactly graphically impaired

Aros desktop, not exactly graphically impaired

Now this year’s winner of “most interesting and promising operative system” must be Aros. It’s a fantastic little OS with plenty of features, and almost as interesting as the operative system itself is it’s exciting and rich background. How do you know that users love their OS? When they decide to reverse engineer it from scratch after the owner goes belly-up, just to make sure it survives (!).

Yes, that was the case with Aros; It’s actually an old-timer in the group of “new operative systems” because Aros is short for “Amiga research operative system”. Yes you read right, Amiga, as in the 1983-2001 prodigy home computer. Odds are that you owned one of these machines in your teens or at least heard of it, and if that is the case then you most likely remember Amiga OS. This was a wonderful operative system lightyears ahead of it’s time. It actually was technically superior to Microsoft, IBM and all the other players back then with around 12 years. That it’s potential was never realized ultimately came down to some spectacularly bad business decisions at Commodore. Never let people who cant code run a computer company.

For my part I remember Amiga OS vividly. My best friend at the time owned an IBM PC and while he was stuck in DOS playing Larry Leasure suite, I was doing things he could only dream about under Amiga OS. I actually learned all my programming skills on the Amiga, so I was an above and beyond fan of the system throughout my teens and well into college.

It was only when Windows 95 came along with it’s fancy new driver system, fast graphics processor and soundblaster cards that the tables were turned. But for roughly 13 years (in Norway at least) the Commodore Amiga was the undisputed champion; both operative-system wise and game/multimedia wise. It was cheap, efficient and the #1 creativity machine. In fact Apple only recently implemented support for REXX which is a scripting system Amiga OS supported from 1990 and beyond (version 1.3 of the software). That speaks volumes about how far ahead Commodore was.

Ok, but what can Aros offer?

Classic Amiga OS version 4

Classic Amiga OS version 4

If you are expecting a boring, grey desktop and faithful reproduction of the early 90’s then no, that is not what Aros is about. Aros is reverse engineered from the Commodore Amiga OS formula; based on available API documentation. As such it’s designed to be compatible on API level, not binary compatible with ancient 16 bit software. In short this means that you can take an Amiga C source-code for instance and re-compile it to run under Aros. But Aros really represents something much more valuable than retro flirting: It represents an actual future for an OS which delivered multitasking a desktop environment 10 years before Windows 3.11 was invented.

The reason Amiga OS has survived all these years is because it’s underlying architecture is brilliant. The way drivers work, the layout of the file-system, how libraries are loaded, how memory is managed, how multi-tasking is solved — all of these things are dealt with in a way which makes for a fast, responsive and highly enjoyable user experience.

It’s not like Windows where evolution meant patch after patch after patch. Amiga OS was designed to avoid bloat. It was designed to deal with libraries and extensions gracefully and intelligently. It was designed to boot fast, respond fast and deliver maximum performance with ridicules sparse resources. Amiga OS boots happily into as little as 4 megabytes of RAM. Yes you read right, four megabytes (not gigabytes). And the speed is phenomenal, you wont believe it until you try it.

Aros is absurdly fast

Aros is absurdly fast

So, with Commodore long gone and the copyright holders showing no interest in doing anything with the Amiga brand, Aros is ultimately the last hope of a realization of this 30 year old dream. A dream of a small, compact, fast and “no bloat” operative system performing miracles from pennies and scraps of ram and CPU. And it works brilliantly. The formula which Carl Sassenracht (the authorof large parts of the OS) deviced all those years ago has proven to be the software equivilent of the ARM processor. ARM was likewise an “old timer” technology until recently. It was almost shelved and taken off the market, it was just by accident that they discovered that the ARM architecture could run just as efficient on low-voltage as it did on “ordinary” power. Had it not been for that economic feature, ARM RISC cpu’s would be a thing of the past.

The Amiga is just like that. An older formula and architecture of an operative system, but one which delivers so much computing power with absurdly few resources. Imagine what Aros can be with a few years of development behind it, and perhaps a couple of hundred programmers?

Have you ever wondered just how fast your PC really is? Imagine how fast Windows would be if it was hand-coded in object pascal and assembler. Not a single procedure wasted and no bloat what so ever. Well with Aros you will find out, or at least as close as you can get. Aros is written in C/C++ (like most operative systems), but the architecture and philosophy ensures that it remains small, efficient and lean. My Aros PC boots in around 3 seconds from powering it on until I can use the desktop. And this is on older hardware (my test machine was a 8 year old dual-core system with 4 gigabytes ram). The desktop is snappy and things like moving windows around, playing video — it’s so bloody fast you will hardly believe it.

To be honest, if Aros keeps this up it will turn into a money saver. Windows and Linux has gained weight over the years, both in complexity and media integration. Aros represents in many ways a clean start, a start with a vision which is now 30 years old. A dream of an OS so lightweight, elegant and easy to use that it makes computing fun again. An OS where efficiency matters and everyone tries to tweak the last drop of CPU out of their code. A system where code is regarded as a work of art.

What about software?

This is why I have given Aros a Mickey rating (both winner and loser), because software will always be a new operating system’s achilles heel. But it’s really not as bad as you think. It’s millions of miles beyond obscure systems like Risc OS (recently re-surfacing and enjoying a renaissance on the Raspberry PI). Why? Well, first of all the Aros authors have provided tools for porting from Linux free for all. Secondly, they have made a few extra libraries to facilitate porting to the platform — and third, because Aros can run 68k (Motorola 68000 CPU) based software!

Loads of juicy titles for Aros, but more is needed

Loads of juicy titles for Aros, but more is needed

That’s right, Aros is capable of running almost all classic 16 bit Amiga software through UAE integration. And believe you me, 20 years of coding makes for a rich and mighty collection of software titles. But yes, this large resovare of software is older. So if you are expecting Adobe dreamweaver, mono developer or Photoshop — that’s not going to happen any-time soon.

But if you can live with The Gimp, like millions of Linux users do — then you will find Aros well stocked in the graphics department.

Software development

And now for the biggie. Does Aros support object pascal? That is the mother of all questions isnt it, at least for us Delphi, SMS and FPC developers. There is no point using a platform we cant program for right?

Well I am pleased to say that Aros does support object pascal and there is a full port of freepascal for the platform (!)

Lazarus is still being ported, so it's not quite ready

Lazarus is still being ported, so it’s not quite ready

It must be underlined that Lazarus is still being ported (!), only FPC is ready out of the box, but lazarus will be excellent on this platform! Lazarus feels more at home with Linux due to it’s multi-window layout. Well, Aros is based on Amiga OS which means you can run applications in their own separate screen. And in such an environment Lazarus really comes into it’s own. It’s the undisputed king on Linux, making mono/C# look like a sad toy in comparison.

Using Aros as a real desktop

Before you sell your house and donate your mother to science, you should give Aros a proper test-drive. You dont want to wipe you Windows installation without really knowing what you get. I recommend installing Aros into a VMWare image first (or some other x86 free emulator from Sun) just to get to know it properly. Remember to create two partitions, one boot and one work (larger). This is more or less traditional for Amiga systems.

Is it ready to be the default desktop? Soon

Is it ready to be the default desktop? Soon

To get a hold of software you point your browser at Aminet, which is the #1 source of 68k and more modern software. There you will find classic stuff like PPaint, Deluxe Paint and ofcourse, freepascal. Exploring the Aros website is also a good idea.

Like all small communities (just like the Delphi community) there are forums you can become a member of. This is perhaps a good start so you have someone to ask and help you out. But Aros / Amiga OS is so much simpler than any OS you have tried to date, so once you “get it” you will be able to navigate most of it without help.

Entering a platform with a bang

The utter upside of any platform is that there will be a huge demand for software. In fact, the worst platforms to create software for is in many ways Windows and OS X, because there you will find hundreds of different alternatives doing the same thing.

On a new, fresh OS the competition will be weaker and you stand a good chance of making some cach once the user-base has brown beyond 10.000. I have no idea how big the Aros world is, but I do know there are at least 10.000 Amiga users in the world, be they retro-gamers or die-hard Amigans. The Amiga is actually a phenomenon. It’s 30 years since it was created this summer; and it went out of production around 1999 (Commodore filed for bankryptsy in 1994, but the license to produce and sell Amiga lingered on for many years). Yet still there is a living market for these machines. These are 30 year old computers running at 7 Mhz (not Ghz, but Mhz), yet people love them to death and still use them in their thousands!

It's an impressive feat, remember this is a user project!

It’s an impressive feat, remember this is a user project!

It should be mentioned that a CPU frequency comparison really means little for the Amiga. It has a set of custom chips dealing with memory copying, pixel moving and everything else. Also the motorola CPU is very different and can do a lot of stuff intel cant with few instructions.

The Amiga was ultiately beaten technically by the lack of single-datatype pixels (e.g: 1 byte for 8bit, 2 bytes for 15/16 bit, 3 bytes for 24 bits and 4 bytes for 32 bit graphics) like the PC have. The Amiga had something called inter-leaved graphics which means a single picture is divided into X layers at different places in memory. For a 32 color display you actually have to write to 5 different memory addresses to set a single pixel (!)

In short, the Amiga has to work 5 times as hard to deliver the same as a PC. Yet you will see these old Amiga computers knock out effects and demos which are en-par with modern PC demos. It really is a strange and mysterious social phenomenon, shrouded in mystery and lore — the platform just wont die! It’s almost ridicules how much beating this system takes, yet there it stands — a reminder of what could have been. An alternative reality which could have been ours where operating systems were light, fast and fun – as opposed to big, bloated and expensive.

Freepascal on Aros

One of the things I love about object pascal no matter the dialect, is that just like the Amiga – object pascal is a language which just wont go away. People have been predicting the downfall of object pascal for decades now, yet object pascal is still highly popular and very much in demand around the world. And just like the Aros team reversed engineered Amiga OS from scratch, so did the FPC and Lazarus team do with Delphi. They cloned a free, open-source version of a language and development platform they loved – to ensure that it would survive regardless of it’s legal owner’s whims.

Older freepascal running under AROS

Older freepascal running under AROS

The result today is that object pascal has a huge, staggering, overwhelming public domain library of source code. You will find units and libraries for just about every conceivable topic. From super-fast games to serious and excellent tools — object pascal have it all.

This means that with a bit of porting work, replacing OS calls where possible – or re-implementing stuff in pure object pascal, thousands of titles could be implemented with ease on the Aros platform. A person with skill and a bit of time on his/her hands can really make a nice bit of money here, by charging a small fee for each program and being honest and give support, the Aros platform would grow rapidly — and your income with it.

Final verdict

Hm. This is always the tricky part.

Aros is slowly becoming a complete desktop environment. I think it’s important to separate OS from desktop, because they are essentially two different things. The OS is what makes a desktop possible, what makes windowing possible, what makes everything possible. A desktop is the human interface into those features- typically provided through default programs and behavior.

Aros is, shortly said, a pretty faithful re-implementation of Amiga OS. But it has evolved away from the old Amiga look and feel (thankfully). But while the underlying OS is no doubt up for the challenge, the desktop experience and default software doesnt quite impress. At least not users who havent started an Amiga in 20 years, or people who have never used an Amiga at all.

Having said that Aros is “almost there”. It’s missing a few minor applications which will add the final touches, but most of all it’s missing that “killer application” to make it popular. For Microsoft it was Word that started it all, for Mac it was no doubt Photoshop, Logic and pro-tools (counting from when Steve Jobs came back up until now), followed by the “family package” which is iMovie, iTunes, iThis and iThat. You get fantastic apps with your mac and that draws customers in.

What Aros really needs is a fantastic software package which only exists on that platform. What exactly that should be I leave to you, the reader. As object pascal programmers and armed with the might and onslaught of our collective source-code repositories, I have no doubt what so ever that it can be object pascal that delivers this — but it will require hard work, dedication and a vision.

If Linux is not your thing — head over to The AROS website and have a look — and yes, it’s free (!)