Archive
QuartexDeveloper.com is now active
It’s taken a while but Quartex Pascal now has it’s own website and forum. You can visit QuartexDeveveloper.com and check it out.
The SSL certificates are being applied within 72hrs according to the host, so don’t be alarmed that it shows up under HTTP rather than HTTPS right now – that is just temporary.
Up until now we have operated with a mix of donations and Patreon to help fund the project, but obviously that model doesn’t scale very well. After some debate and polls on the Facebook group I have landed on a new model.
Funding and access model
Starting with the release of version 1.0, which is just around the corner – the model will be as such:
- Backing and support will be handled solely through Patreon
- Facebook group will become open for all
- Patreon tiers will be modified to reflect new model
- Main activity and news will shift to our website, quartexdeveloper.com
- Community build will be available from our website
- Commercial license will also be available from our website
So to sum up, the following 3 options are available:
- Back the project on Patreon, full access to the latest and greatest 24/7
- Community edition, free for educational institutions and open-source projects (non commercial)
- Commercial license is for those that don’t want to back the project on a monthly basis, but instead use the community edition in a professional capacity for commercial work.
With the community edition available, why should anyone bother to back the project you might ask? Well, the public builds will by consequence be behind the latest, bleeding edge builds since the community edition is only updated on minor or major version increments (e.g. when version changes from 1.0 to 1.1). Users who back the project via Patreon will have instant access to new documentation, new packages with visual components, new project templates, RTL fixes and patches as they are released. These things will eventually trickle down to the community edition through version increments, but there is a natural delay involved.
This is how most modern crowd funded projects operate, with LTS builds (long term support) easily available while the latest cutting edge builds are backers only. Documentation, fixes and updates to components, new component packages, hotfixes and so on – is the incentive for backing the project.
This is the only way to keep the ball rolling without a major company backing day to day development, we have to get creative and work with what we got. Projects like Mono C# had the luxury of two major Linux distribution companies backing them, enabling Miguel de Icaza to work full time on the codebase. I must admit I was hoping Embarcadero would have stepped in by now, but either way we will get it done.
Onwards!
Quartex Pascal: Public Alpha
It’s been a long time since I have written a post here on my blog. I have been so busy with work that I quite frankly have not had the extra energy to maintain this website. During the weekdays my hands are full with work, and in the weekends I typically recharge 1 day, with Friday afternoon and Sunday allocated for the Quartex Pascal project. On Saturdays I sleep for as long as I can, go for a walk, and watch Netflix.
Thankfully I am happy to report that Quartex Pascal is more or less ready for a public alpha. I took out a week vacation today to finish the remaining handful of tickets, which are ultimately superficial and fiddly, but nothing difficult compared to what we have already achieved.
Have we reached our goals?
When you start a project it’s easy to get caught up in the potential. One feature quickly avails the next, and if you are not careful – you can be whisked away to vaporware land. Or even worse, end up with a project that never ends and where you keep telling yourself “i just need to add this, then I’m done”. I am happy that we have managed to avoid that, and set a clear boundary of what should be in the initial release. The point of version 1.0 is not to cover all possible features, but rather – to make damn sure the fundamental features work as well as I can make them. Because future revisions and features will build on that foundation. So in short: Yes. As much as 90% of what we set out to include in version 1.0 has been realized. The only thing we had to push to version 1.1 and 1.2 is the database explorer, DAC classes so your web application can work directly with a database through a node.js service, and a few minor features like having a Gr32 powered picture viewer and paint-program included.
There is plenty of room for optimization and refactoring in the code-base, so once the first version is out you can expect regular updates (Patreon backers only) where both the IDE and RTL becomes more and more refined and optimized. One of the things I am really looking forward to is writing new and exciting widgets (controls are called widgets under the QTX paradigm), and also port over more JS modules and frameworks. We already have an impressive list of JS frameworks that you can use out of the box. The benefit of having highly skilled backers is that they are quick to digest new technology and produce packages, so we have a lot of widgets that you can drag & drop that are 1:1 wrappers (a wrapper is a class definition that describes an external object, or a unit that makes the features of an external framework usable from pascal).
The Cloud desktop project
Since a couple of years have passed, most people have probably forgotten why Quartex Pascal was created to begin with. Namely as a development tool to implement and finish the Quartex Media Desktop (also known as Amibian.js). Quartex Pascal was actually a detour we had to make to save the codebase I had already implemented. So the moment version 1.0 goes out the door, my first priority is to refactor and re-implement the desktop client under the Quartex Pascal RTL. The background node.js services already run on my new RTL, so all the work we did a couple of years ago is still there, waiting to be picked up again.
I am not going to spend ages re-hashing the desktop system, but in short this is a client-server system that implements a Windows like desktop, complete with filesystem over websocket, multi user accounts, message based API, and that is 100% JavaScript from the back-end services all the way up to the desktop itself. It is in other words portable and completely hardware and platform agnostic. The point of the desktop is to provide the exact same ecosystem that Windows provides for native applications, for enterprise level web applications. This includes hybrid application modules where half the program is deployed server-side, while the visual part is rendered in the browser (this is how we could have a Torrent client with live status in a web application).
Combine this with a thin Linux bootstrap, where you boot into Chrome in Kiosk mode – and you have a fully working, incredibly powerful desktop system. One that you can literally copy from one machine to the next without recompiling a line of code. As long as the system supports node.js and have a modern browser, Amibian.js will run. Heck, I even booted it on my Smart TV (!).
Release date?
The public alpha is, as the name implies, a pre-release version meant purely to be played around with. There are bound to be hiccups and bugs, but the point is just to get you familiar with the ecosystem (which is very different from Delphi, so don’t think you can just magically compile some old Delphi application).
I am aiming at next weekend. It can be that some delay comes up, but all in all I have only a handful of tickets, most of them small and somewhat fiddly, but nothing too difficult. I will close 2 or 3 tickets a day, so a build should be ready next weekend for you guys.
What License?
The application is released as vanilla shareware, which means that copyright and ownership of all materials (except packages and examples written by others naturally) is tied to me. Once we have enough to establish the Quartex Pascal Foundation (which aims at teaching object pascal and offering free development tools for for students, schools and non-profit organizations) ownership will be isolated there. You can read more about the license on the website, here: https://quartexpascal.wordpress.com/about/licensing/.
We also have a rule that any version of Quartex Pascal will never cost more than €300. The first version will be in the €100-€150 range, which buys you a license to use the development tools in a commercial setting. Quartex Pascal is free to use for open-source work. Students can also use QTX for free, provided they provide proper student identification that can be verified, and they dont use it for commercial gain. Considering the cheap price, buying a license wont exactly break the bank.
The RTL is accessible the exact same way that you are used to under Delphi, so exploring the RTL is encouraged. I have started on the documentation but this is an alpha so you really need to explore a bit.
Just like Delphi all applications have a TApplication object that is the first to be created, and entities like forms are managed by TApplication (they automatically register when you create TQTXForm or TQTXWindow). Once you familiarize yourself with the units, you should have no problem becoming productive in a very short time.
Resources
- Check out the Patreon project page for backing this project
- Check out the Quartex Pascal website
- Check out the Quartex Pascal usergroup on Facebook (backers only!)
When Nicola Tesla slaps you over the head
If you have poked around the Quartex Pascal RTL you might have notice that QTX comes with a serial-number minting system. Creating unique serial numbers that can be mathematically re-traced back to a root key (number sequence) is extremely tricky business. My solution was to dip my fingers into esoteric and occult numerology, because our ancient forbearers had a knack for puzzles and number based riddles.
And here I thought I was super clever, only to discover that Nicola Tesla scribbled a similar system on a napkin back in the late 1800s (figuratively speaking). Turns out that the basis of my system is more or less identical to Tesla’s numbers and ultimately bound by their relationships, where you operate with a growth factor that is a multiple of 12, modulated and held in check by Fibonacci, Lucas or Leonardo sequencing.
So my ego got a well deserved slap (which is always healthy, we should all be humble in the presence of that man).
I have never really been that interested in Tesla until recently, and the little I have read about him makes me incredibly sad. This man was not decades ahead of his time, but centuries.
In my view, the biggest tragedy in human history is without a doubt the loss of the great library in Alexandria, Egypt. Second only with the murder of Hypatia; a series of events that would eventually catapult humanity as a whole into a dark-age that lasted for 2000 years.
But having spent some time this morning reading about Tesla, I would add him to that list of tragic events that have affected our history (or in his case, being prevented from lifting mankind up). This is a man that constructed the walkie-talkie in the late 1800s. He even theorized that both audio and video could be transmitted over a hand-held device. And this was in the late 1800s (!).
Above: The serial-number minting dialog from the IDE. Here we use 12 seed numbers to form the root key, and each serial number is grown from these using natural numbers, as employed by various mystics and esoteric traditions.
Hat off Tesla. It is a great shame that you were born into a world that neither understood or appreciated the wonders you delivered.
Nicolas Tesla’s notebooks is best read on your knees.
Now I need to scotch tape my ego back together and get to work.
HTMLComponents for Delphi, a closer look
For a while now I have been paying attention to Alexander Sviridenkov’s components for Delphi. First of all because Delphi doesn’t really have that many options when dealing with HTML beyond embedding the classical browsers (Chromium, Firefox or Edge); And while Dave Baldwin’s work has served the community well in the past, TFrameBrowser is 20 years old. So there is definitively room for improvement.
Secondly, in my work with Quartex Pascal, a system that compiles Object Pascal to JavaScript for HTML5, being able to work with HTML and render HTML easily from my IDE is obviously interesting. The form designer in particular could benefit from showing “live” graphics rather than representative rectangles.
All of that is possible to do with Chromium if you run it in an off-screen capacity, but getting good results is very tricky. Chromium Embedded runs in a separate thread (actually, multiple threads) and sharing video memory, injecting HTML to avoid a full reload — lets just say that a Delphi native component package would make all the difference. Enter HTMLComponents.
Focus on the essentials first
The way that Alexander has proceeded with his components can resemble my own philosophy (or indeed anyone who has been a developer for a while). It’s the kind of work approach you end up with through experience, namely, to start with the basics and make sure that is rock solid (read: focus on the foundation code, that’s what’s going to matter the most. Trust me). It’s so tempting to run off on a tangent, adding more and more functionality – typically visually pleasing stuff, but as mature developers will know, if you go down that path what you end up with is a very expensive mess.
Thankfully, Alexander has gone about his work in an orderly, no-nonsense way. He began with the HTML parser, making sure that was flexible, compliant and delivered great performance (over 100 Mb a second!). Then he moved on to the CSS part of the equation and implemented a high performance styling engine. The reason I outline this is because I don’t think people fully grasp the amount of work involved. We think of HTML as a simple tag based format, but the sheer infrastructure you need to represent modern HTML properly is enormous. There is a reason Delphi component vendors shy away from this task. Thankfully Alexander is not one of them.
Scripting?
Next we have the scripting aspect. And here is the twist, if we can call it that. HTMLComponents is not written to be a browser. It is written to enable you to render HTML5 at high speed within a native application, including CSS animations and Tweening (a technique made popular by Apple. Like sliding forms or bouncing swipe behavior).
In other words, if you are expecting to find a browser, something like Dave Baldwin’s now ancient TFrameBrowser, then you should probably look to the new TEdgeBrowser component from Embarcadero. So JavaScript is not yet natively supported. HTMLComponents falls into the category of a UI presentation framework more than a browser.
If however, like myself, you want to handle presenting HTML5, PDF, RTF and Word documents without a ton of dependencies (Chromium adds 150Mb worth of libraries you need to ship), provide your users with a compliant HTML WYSIWYG Editor – and also deliver those fancy animated UI elements – then you are going to love HTMLComponents.
I should mention that HTMLComponents has its own scripting engine, but it’s not JavaScript. But for those situations where a script is needed, you can tap into the scripting engine if you like. Or deal with everything natively. It’s your choice.
Document editor
The reason I mentioned Alexander’s architecture and how his codebase has evolved, is because a high performance document rendering engine can be very useful for a variety of tasks.
One thing is rendering HTML5 with all the bells and whistles that entails, but what about RTF? What about Word documents? What about PDF documents? Once you have a rock solid engine capable of representing HTML5, the next logical step is to branch out and work with the other formats of our times. And that is just what Alexander did.
But before we look at those features, let’s have a peek at what components you get.
As you can see from the picture above, HTMLComponents is not just about drawing HTML. Being able to represent HTML is useful in a variety of situations since it simplifies visual feedback that would otherwise be very time consuming to implement. So instead of limiting that power to a single control, HTMLComponents come with common controls that have been infused with superior graphical powers.
The most powerful component in the above list is without a doubt the HTML editor component (also notice that the package installs both standard and DB variations of the various controls). This is quite simply a fully compliant WYSIWYG editor – complete with all the formatting features you expect.
- WYSIWYG editing.
- Does not use IE or other libraries (100% native Delphi code).
- Supports all Delphi versions from Delphi 5 to Delphi 10.4 Sydney.
- Supports Lazarus (Windows/Linux)
- VCL (Win32/64) FMX (Windows / OSX / Android / iOS / Linux)
- Full support for touch-screen devices – gestures, text selection (Windows Tablets/Screens, iOS, Android, OSX)
- Smooth scrolling on Android and iOS.
- Unicode support for Delphi 6 – 2007 (requires TNTUnicode).
- Scalable (High DPI support).
- Live spellchecking and autocorrection (built-in support for Addict).
- Live text styles preview (font family,size, color, background).
- RTF and MS Word DOCX Import on all platforms.
- PDF export on Windows, Android, OSX and iOS.
- DB-Aware version
- Full support for HTML tags and CSS properties.
- Full access from Delphi code to DOM and Styles.
- Images, lists, blocks, font styles
- Tables support
- Print and Print Preview
- Embedded Find dialog, Text search, Document Index generation.
- Copy from/paste to MS Word, browsers and other applications
- Embedded Markdown, Pascal and HTML syntax highlighting.
- HTML-based editor controls (HtFontCombo, HtFontSizeCombo, HtColorCombo, HtTableAddCombo, HtBorderCombo, HtTableBorderCombo)
That is a solid list of features, and did I mention you get full source-code?
HTML empowered controls
If you are looking over the list of controls above and expecting to find something like a browser or viewer control, you won’t find it. The closest thing to a HTML viewer is the panel control (THtPanel). It exposes properties and methods to populate it with HTML (as does all the controls), set what type of scrollbars you need (if any), how to deal with links, images and CSS styling – and then it’s a matter of feeding some HTML into the control.
Obviously controls like THtCombobox have behavior that is dictated by the OS, but you can style the child elements (rows for example) being displayed, the border etc. using the full might of HTML5. And yes, you can apply CSS transitions there as well – which is (excuse my french) bloody spectacular!
I mentioned that HTMLComponents were not designed to be a browser replacement, but rather to make it easier for native developers to tap into the design power and visual feedback that makes HTML5 so productive to use for UIs. Well, once you have set the properties for a panel and given it some HTML -you can do some pretty amazing things!
HTML takes a lot of grunt work out of the equation for you. For example, let’s say you wanted to produce a demo like the one in the picture above (see direct link in the next paragraph). With all the effects, transitions, pictures and displacement elements. Just how much work would that be in traditional Delphi or C++ ?
Note: You can download the Demo application directly, here:
https://delphihtmlcomponents.com/csstransforms.zip
First you would need a panel container for each picture, then a canvas to hold the picture, then you would need to handle the interaction events- and finally access the canvas to draw the many alpha blended transitions (the picture here really doesn’t do the framework credit, you have to see them to fully appreciate the level of detail and performance HTMLComponents delivers). And all of that is before you have new elements flying in from the sides or above, that fades perfectly with the backdrop. All of it working according to a timeline (tweening as its called).
Instead of all that work, having to write a tweening engine, 32 bit alpha-blending DIBs (device independent bitmaps), deal with god knows how much work — you can just deliver some HTML and CSS and let HTMLComponents handle all of it. With zero external dependencies I might add! This is a pure Delphi library. There are no references to external browser controls or anything of the kind. HTMLComponents does exactly what it says on the box – namely to render HTML5 at high speed. And it delivers.
Here is the HTML for one of the pictures with effects in the demo:
<div class="view view-sixth">
<img src="images/13.jpg" />
<div class="mask">
<h2>Hover Style #6</h2>
<p>A wonderful serenity has taken possession ..</p>
<a href="#" class="info">Read More</a>
</div>
</div>
And here is the CSS animation transition code for the same. Please note that the original code contained definitions for IE, Opera, Webkit and Firefox. I removed those for readability:
.view-sixth img {
transition: all 0.4s ease-in-out 0.5s;
}
.view-sixth .mask {
background-color: rgba(146,96,91,0.5);
filter: alpha(opacity=0);
opacity: 0;
transition: all 0.3s ease-in 0.4s;
}
.view-sixth h2 {
filter: alpha(opacity=0);
opacity: 0;
border-bottom: 1px solid rgba(0, 0, 0, 0.3);
background: transparent;
margin: 20px 40px 0px 40px;
transform: scale(10);
transition: all 0.3s ease-in-out 0.1s;
}
.view-sixth p {
filter: alpha(opacity=0);
opacity: 0;
transform: scale(10);
transition: all 0.5s ease-in-out 0.2s;
}
.view-sixth a.info {
filter: alpha(opacity=0);
opacity: 0;
transform: translateY(100px);
transition: all 0.3s ease-in-out 0.1s;
}
.view-sixth:hover .mask {
filter: alpha(opacity=100);
opacity: 1;
transition-delay: 0s;
}
.view-sixth:hover img {
transition-delay: 0s;
}
.view-sixth:hover h2 {
filter: alpha(opacity=100);
opacity: 1;
transform: scale(1);
transition-delay: 0.1s;
}
.view-sixth:hover p {
filter: alpha(opacity=100);
opacity: 1;
transform: scale(1);
transition-delay: 0.2s;
}
.view-sixth:hover a.info {
filter: alpha(opacity=100);
opacity: 1;
transform: translateY(0px);
transition-delay: 0.3s;
}
If CSS is not something you normally don’t bother with, the code above might look complex and alien. But there are tons of websites that have wizards, tutorials and even online editors (!), so if you take the time to read up on how CSS transitions work (they are quite easy), you will knock out some impressive effects in no time.
Once you have built up a collection of such effects, just link it into your Delphi application as a resource if you don’t want external files. Personally I think its a good thing to setup the UI in separate files like that, because then you can update the UI without forcing a binary installation on your customers.
So if we consider the amount of Delphi code we would have to write to deliver the same demo using stock VCL, sum up the cost in hours – and most likely the end result as well (Alexander is exceptionally good at graphical coding), I for one cant imagine why anyone would ignore HTMLComponents. I mean serious, you are not going to beat Alexander’s code here. And why would you waste all that time when you can buy ready to use controls with source-code for such a modest price?
Office formats
I mentioned briefly that with a powerful document rendering engine in place, that the next step of the way would be to cover more formats than just HTML. And this is indeed what Alexander has done.
If you invest in his Add-On Office package for HTMLComponents, you will be able to load and display a variety of document formats. And just like HTMLComponents the code is 100% Delphi and has zero dependencies. There are no COM objects or ActiveX bindings involved. Alexander’s code loads, parses and converts these documents instantly to HTML5, and you can view the results using HTMLComponents or in any modern browser.
Following document formats are supported:
- Rich Text Format (RTF)
- MS Word 6-2007 binary format (DOC)
- MS Word XML document (DOCX)
- MS Power Point binary format (PPT)
- MS Power Point XML format (PPTX)
- MS Excel binary format (XLS)
- MS Excel XML format (XLSX)
- Adobe PDF format (PDF)
- Supercalc format (SXC)
- EPUB (electronic books).
Besides the document conversion classes you also get the following code, which is pretty hard-core and useful:
- EMF/WMF to SVG conversion
- TTF to WOFF conversion
- TTF normalization
- TTF to SVG conversion
- CFF to TTF conversion
- Adobe PostScript to TTF conversion.
For me this was a god-send because I was using Gnostice’s PDF viewer to display the documentation for Quartex Pascal in the IDE. Being able to drop that dependency (and cost!) and use HTMLComponents uniformly throughout the IDE makes for a much smaller codebase – and cleaner code.
Final thoughts
The amount of code you get with HTMLComponents is quite frankly overwhelming. One thing is dealing with a tag based format, but once you throw special effects, transitions and standards into the mix – it quickly becomes a daunting task. But Alexander is delivering one of the best written component packages I have had the pleasure of owning. If you need a fresh UI for your application, be it POS, embedded or desktop utilities – HTMLComponents will significantly reduce the time spent.
I should also underline that HTMLComponents also works on FMX and Mobile devices ( Windows, OS X, Android, iOS and Linux even!). I’m not a huge fan of FMX myself so being able to design my forms using HTML and write event handlers in native Delphi is perfect. FMX has a lot of power, but the level of detail involved can be frustrating. HTMLComponents takes the grunt out of it, so I can focus on application specific tasks rather than doing battle with the UI.
The only thing I would like to see added, is support for JavaScript. HTMLComponents makes it easy for you to intercept scripts and deal with them yourself (HTMLComponents also have a pascal inspired script), but I do hope Alexander takes the time to add Besen (a native Delphi JavaScript engine) as an option. It really is the only thing I can think of in the “should have” apartment. Everything else is already in there.
I have to give HTMLComponents 9 out of 10 stars. It would have scored a perfect 10 with JS support. But this is the highest score I have ever given on my blog, so that’s astronomical. Well done Alexander! I look forward to digging into the office suite in the weeks ahead, and will no doubt revisit this topic in further articles.
Visit Alexander’s website here: https://www.delphihtmlcomponents.com/index.html
Quartex Pascal Build 13b ships
While it can come across as disingenuous, I frickin love this project! As a developer yourself you know that feeling, when you manage to unify various aspects of your program, so that it all fits just perfectly. And the way I implemented file-handling and editors is within that sweet spot.
What is new?
It’s been a couple of weeks since I posted here last, so the list of changes will be quite dramatic. I think its best to focus on the highlights or this post would become very long!
Ironwood license management
Up until 2018 one of my products was a component package called HexLicense. This is a component package for Delphi that provides serial number validation, license handling and (most importantly) serial number minting. The HexLicense components were sold commercially until 2018 when I took them off the market and open-sourced (access was via Patreon. It is now moved to the Quartex Pascal project instead).
Im not going to go into how difficult it is to produce thousands of distinctly different serial numbers based on seed data, but it’s no walk in the park.
The final implementation I made for license minting and validation, was called Ironwood. It took the engine behind HexLicense and took it to a completely new level, incorporating both obfuscation and number modulation.
Needless to say, Ironwood is now a part of the Quartex Pascal RTL. To make it easier to work with the IDE has a nice utility for generating license-numbers, loading and saving keys, exporting license number batches – and much more.
There is also a ready-to-rock node.js application that can generate keys from the command-line (which is good to invoke from a server or service, so that it executes as a separate process).
HTML structure provider
The IDE has a very clean internal architecture, where the actual work is isolated in a set of easy to understand classes. One of these classes is called a TIDEAstProvider class. This is a class whose job it is to parse and otherwise work with whatever content an editor has, and deliver symbolic information that can be displayed in the file-structure treeview.
Obviously we have an object pascal provider, which will quickly compile and generate an AST very quickly in memory. This is used to power both the structure treeview and the code-suggestion.
Next, we have the exact same provider for JavaScript. So when you open a JavaScript file, the file will be processed to produce an AST, and the symbol information will be displayed exactly like your object pascal is. So behavior between these are identical.
We now also have a HTML provider, with a CSS provider on the way. The HTML provider is still in its infancy, but its flexible enough to represent a good foundation to work with. So I will no doubt return to this task later to smarten the provider logic up, and better handle un-valid HTML and CSS.
Code suggestion
Code suggestion is a pretty standard function these days. We have had support for this under Object Pascal for a while now in the IDE (with JavaScript on the way).
Note: the code suggestion-box is un-styled at this point. Custom painting will be added once the core functionality is done.
Code suggestion for HTML is now in place too. It needs a bit of polish since the rules for HTML are wildly different from a programming language, but common behavior like TAG suggestion is there — with attributes, properties and events to follow.
So even if you are not an object pascal developer, the IDE should be nice to work with for traditional JavaScript / HTML code.
Form Recognition
While we cannot activate the form-designer just yet, since we need more AST functionality to extract things like class properties and attributes “live” to be able to do that properly — we are getting really close to that milestone.
The IDE however now recognize form files, so if your unit has an accompanying DFM file, the IDE is smart enough to open up a form-page. Form pages are different from ordinary pascal pages, since they also have the form designer control on a sub-tab. More or less identical to Delphi and Lazarus.
It is going to be so nice to get the form-designer activated. Especially the stack-based layout, which makes scalable, dynamic layout easy to create and work with.
The QTX RTL also supports orientation awareness as a part of the visual component system. One of the first things you will notice when exploring the code, is that ReSize() ships in an Orientation parameter, so you can adjust your layout accordingly.
Help and documentation inside the IDE
The IDE now has a PDF viewer with search functionality built-in. So when you click on Help and Documentation, a tab which shows the documentation PDF opens. This makes it easy to read, learn and find the information you need fast.
Well, that was a brief overview of what has changed since last time!
Next update is, as always, the weekends. We tend to land on sundays for new binaries, but do issue hotfixes in the evenings (weekdays) if something critical shows up.
Come join the fun!
Want to support the project? All financial backers that donates $100+ get their name in the product, access to the full IDE source-code on completion, and access to the Quartex Media Desktop system (which is a complete web desktop with a clustered back-end, compiled to JavaScript and running on node.js. Portable, platform and chipset independent, and very powerful).
A smaller sum monthly is also welcome. The project would not exist without members backing it with $100, $200 etc every month. This both motivates and helps me allocate hours for continuous work.
When the IDE is finished you will also have access to the IDE source-code through special dispensation. Backers have rights since they have helped create the project.
All donations are welcome, both large and small. But donations over $100, especially reoccurring, is what drives this project forward.
Remember to send me a message on Facebook so I can add you to the Admin group: https://www.facebook.com/quartexnor/
Quartex Pascal, Build 10b is out for backers
I am deeply moved by some of the messages I have received about Quartex Pascal. Typically from people who either bought Smart Mobile studio or have followed my blog over the years. In short, developers that like to explore new ideas; people who also recognize some of the challenges large and complex run-time libraries like the VCL, FMX and LCL face in 2020.
Since I started with all this “compile to JavaScript” stuff, the world has changed. And while I’m not always right – I was right about this. JavaScript will and have evolved into a power-house. Largely thanks to Microsoft killing Basic. But writing large scale applications in JavaScript, that is extremely time consuming — which is where Quartex Pascal comes in.
Quartex Pascal evolves every weekend. There are at least 2 builds each weekend for backers. Why not become a backer and see the product come to life? Get instant access to new builds, the docs, and learn why QTX code runs so much faster than the alternatives?
A very important distinction
Let me first start by iterating what I mentioned in my previous post, namely that I am no longer involved with The Smart Company. Nor am I involved with Smart Mobile Studio. I realize that it can be difficult for some to separate me from that product, since I blogged and created momentum for it for more than a decade. But alas, life change and sometimes you just have to let go.
The QTX Framework which today has become a fully operational RTL was written by me back between 2013-2014 (when I was not working for the company due to my spinal injury). And the first version of that framework was released under an open-source license.
When I returned to The Smart Company, it was decided that to save time – we would pull the QTX Framework into the Smart RTL. Since I owned the QTX Framework, and it was open source, it was perfectly fine to include the code. The code was bound by the open source licensing model, so we broke no rules including it. And I gave dispensation that it could be included (although the original license explicitly stated that the units should remain untouched and separate, and only inherited from).
Quartex Media Desktop, a complete desktop system (akin to X Windows for Linux) written completely in Object Pascal, including a clustered, service oriented back-end. All of it done in Quartex Pascal — a huge project in its own right. Under Quartex Pascal, this is now a project type, which means you can have your own cloud solution at the click of a button.
As I left the company for good before joining Embarcadero, The Smart Company and myself came to an agreement that the parts of QTX that still exists in the Smart Mobile Studio RTL, could remain. It would be petty and small to make a huge number out of it, and I left on my own terms. No point ruining all that hard work we did. So we signed an agreement that underlined that there would be overlaps here and there in our respective codebases, and that the QTX Framework and Quartex Media Desktop was my property.
Minor overlaps
As mentioned there will be a few minor overlaps, but nothing substantial. The class hierarchy and architecture of the QTX RTL is so different, that 90% of the code in the Smart RTL simply won’t work. And I made it that way on purpose so there would be no debates over who made what.
QTX represents how I felt the RTL should have been done. And its very different from where Smart Mobile Studio ended up.
The overlaps are simple and few, but it can be helpful for Smart developers to know about if they plan on taking QTX for a test-drive:
- TInteger, TString and TVariant. These were actually ported from Delphi (part of the Sith Library, a pun on Delphi’s Jedi Library).
- TDataTypeConverter came in through the QTX Framework. It has been completely re-written from scratch. The QTX version is endian aware (works on both ARM, X86 and PPC). Classes that deal with binary data (like TStream, TBuffer etc) inherit from TDataTypeConverter. That way, you dont have to call a secondary instance just to perform conversion. This is easier and much more efficient.
- Low-level codecs likewise came from the QTX Framework, but I had to completely re-write the architecture. The old model could only handle binary data, while the new codec classes also covers text based formats. Codecs can be daisy-chained so you can do encoding, compression and encryption by feeding data into the first, and catching the processed data from the last codec in the chain. Very handy, especially when dealing with binary messages and database drivers.
- The in-memory dataset likewise came from the QTX Framework. This is probably the only unit that has remained largely unchanged. So that is a clear overlap between the Smart RTL and QTX.
- TextCraft is an open source library that covers Delphi, Freepascal and DWScript. The latter was pulled in and used as the primary text-parser in Smart. This is also the default parser for QTX, and have been largely re-written so it could be re-published under the Shareware license.
Since the QTX RTL is very different from Smart, I haven’t really bothered to use all of the old code. Stuff like the CSS Effects units likewise came from the QTX Framework, but the architecture I made for Smart is not compatible with QTX, so it makes no sense using that code. I ported my Delphi tweening library to DWScript in 2019, which was a part of my Patreon project. So most of the effects in QTX use our own tweening library. This has some very powerful perks, like being able to animate a property on any object rather than just a HTML Element. And you can use it for Canvas drawing too, which is nice.
Progress. Where are we now?
So, where am I in this work right now? The RTL took more or less 1 year to write from scratch. I only have the weekends for this type of work, and it would have been impossible without my backers. So I cannot thank each backer enough for the faith in this. The RTL and new IDE is actually just a stopping point on the road to a much bigger project, namely CloudForge, which is the full IDE running as an application on the Quartex Media Desktop. But first, let’s see what has been implemented!
AST unit view
The Unit Overview panel. Easy access to ancestor classes as links (still early R&D). And the entire RTL on a second tab. This makes it very easy to learn the new RTL. There is also proper documentation, both as PDF and standard helpfile.
When the object-pascal code is compiled by DWScript, it goes through a vigorous process of syntax checking, parsing, tokenizing and symbolization (or objectification is perhaps a better word), where every inch of the code is transformed into objects that the compiler can work with and produce code from. These symbols are isolated in what is known as an AST, short for “Abstract Symbol Tree”. Basically a massive in-memory tree structure that contains your entire program reduced to symbols and expressions.
In order for us to have a live structural view of the current unit, I have created a simple background process that compiles the current unit, grabs the resulting AST, locates the unit symbol, and then displays the information in a meaningful way. This is the exact same as most other IDE’s do, be it Visual Studio, Embarcadero Delphi, or Lazarus.
So we have that in place already. I also want to make it more elaborate, where you can click yourself to glory by examining ancestors, interfaces, partial class groups – as well as an option to include inherited members (which should be visually different). Either way, the AST code is done. Just need to consolidate a few tidbits so that each Treeview node retains information about source-code location (so that when you double-click a symbol, the IDE navigates to where the symbol exists in the codebase).
JavaScript parsing and compilation
QTX doesn’t include just one compiler, but three. In order for the unit structure to also work for JavaScript files I have modified Besen, which is an ES5 compatible JavaScript engine written in Delphi. And I use this much like DWScript to parse and work with the AST.
Besen is a wonderful compiler. Where DWScript produces JavaScript from Object Pascal, Besen produces bytecodes from JavaScript (which are further JIT compiled). This opens up for some interesting options. I need to add support for ES6 though, modules and require are especially important for modern node.js programming (and yes, the QTX RTL supports these concepts)
HTML5 Rendering and CSS preview
Instead of using Chromium inside the IDE, which is pretty demanding, I have decided to go for HTMLComponents to deal with “normal” tasks. The “Welcome” tab-page for example — it would be complete overkill to use a full Chromium instance just for that, and TEdgeBrowser is likewise shooting sparrows with a Bazooka.
THTMLComponents have a blistering fast panel control that can render more or less any HTML5 document you throw at it (much better than the old TFrameViewer component). But obviously, it doesn’t have JS support. But we won’t be using JS when displaying static information – or indeed, editing HTML5 compliant content.
WYSIWYG Editor
The biggest benefit for HTMLComponents, is that it’s a fully operational HTML compliant editor. Which means you can do more or less all your manual design with that editor. In Quartex Pascal there is direct support for HTML files. Quartex works much like Visual Studio code, except it has visual designers. So you can create a HTML file and either type in the code manually, or switch to the HTMLComponents editor.
Which is what products like Help & Manual uses it for
Image from HTMLComponents application gallery website
Support for HTML, CSS and JS files directly
While not new, this is pretty awesome. Especially since we can do a bit of AST navigation here too to present similar information as we do for Object Pascal. The whole concept behind the QTX RTL, is that you have full control over everything. You can stick to a normal Delphi like form designer and absolute positioning, or you can opt for a more dynamic approach where you create content via code. This is perfect for modern websites that blend scrolling, effects and content (both dynamic and static content) for a better user experience.
You can even (spoiler alert), take a piece of HTML and convert it into visual controls at runtime. That is a very powerful function, because when doing large-scale, elaborate custom controls – you can just tell the RTL “hey, turn this piece of HTML into a visual control for me, and deliver it back when you are ready).
Proper Form Designer
Writing a proper form designer like Delphi has is no walk in the park. It has to deal not just with a selected control, but also child elements. It also has to be able to select multiple elements based on key-presses (shift + click adds another item to the selection), or from the selection rectangle.
A property form layout control. Real-time rendering of controls is also possible, courtesy of HTMLComponents. But to be honest, it just gets in the way. Its much easier to work with this type of designer. It’s fast, responsive, accurate and will have animated features that makes it a joy to work with.
Well, that’s not going to be a problem. I spent a considerable amount of time writing a proper form designer, one that takes both fixed and dynamic content into account. So the Quartex form designer handles both absolute and stacked layout modes (stacked means top-down, what in HTML is knock as blocking element display, where each section stretch to the full width, and only have a defined height [that you can change]).
Node.js Service Protocol Designer
Writing large-scale servers, especially clustered ones, is very fiddly in vanilla JavaScript under node.js. It takes 3 seconds to create a server object, but as we all know, without proper error handling, a concurrent message format, modern security and a solid framework to handle it all — that “3 second” thing falls to the ground quickly.
This is where the Ragnarok message system comes in. Which is both a message framework, and a set of custom servers adapted for dealing with that type of data. It presently supports WebSocket, TCP and UDP. But expanding that to include REST is very easy.
This is where the full might of the QTX Framework begins to dawn. As i wrote before we started on the Quartex Media Desktop (Which this IDE and RTL is a part of), in the future developers wont just drag & drop components on a form; they will drag & drop entire ecosystems ..
But the power of the system is not just in how it works, and how you can create your own protocols, and then have separate back-end services deal with one part of your infrastructure’s workload. It is because you can visually design the protocols using the Node Builder. This is being moved into the QTX IDE as I type. So should make it for Build 12 next weekend.
In short, you design your protocols, messages and types – a bit like RemObjects SDK if you have used that. And the IDE generates both server and client code for you. All you have to do is fill in the content that acts on the messages. Everything else is handled by the server components.
Suddenly, you can spend a week writing a large-scale, platform agnostic service stack — that would have taken JavaScript developers months to complete. And instead of having to manage a 200.000 lines codebase in JavaScript — you can enjoy a 4000 line, easily maintainable Object Pascal codebase.
Build 11
Im hoping to have build 11 out tomorrow (Sunday) for my backers. Im still experimenting a bit with the symbol information panel, since I want it to be informative not just for classes, but also for methods and properties. Making it easy to access ancestor implementations etc.
I also need to work a bit more on the JS parsing. Under ES5 its typical to use variables to hold objects (which is close to how we think of a class), so composite and complex datatypes must be expanded. I also need to get symbol position to work property, because since Besen is a proper bytecode compiler, it doesn’t keep as much information in it’s AST as DWScript does.
Widgets (which is what visual controls are called under QTX) should appear in build 12 or 13. The IDE supports zip-packages. The file-source system I made for the TVector library (published via Embarcadero’s website a few months back) allows us to mount not just folders as a file-source, but also zip files. So QTX component packages will be ordinary zip-files containing the .pas files, asset files and a metadata descriptor file that tells the IDE what to expect. Simple, easy and very effective.
Support the project!
Want to support the project? All financial backers that donates $100+ get their name in the product, access to the full IDE source-code on completion, and access to the Quartex Media Desktop system (which is a complete web desktop with a clustered back-end, compiled to JavaScript and running on node.js. Portable, platform and chipset independent, and very powerful).
Your help matters! It pays for components, hours and above all, tools and motivation. In return, you get full access to everything and a perpetual license. No backers will ever pay a cent for any future version of Quartex Pascal. Note: PM me after donating so I can get you added to the admin group! Click here to visit paypal: https://www.paypal.com/paypalme/quartexNOR
All donations are welcome, both large and small. But donations over $100, especially reoccurring, is what drives this project forward. It also gets you access to the Quartex Developer group on Facebook where new builds, features etc is happening. It is the best way to immediately get the latest build, read documentation as its written and see the product come to life!
Quartex Pascal, convergence is near
A Quartex Cluster of 5 x ODroid XU4. A $400 super computer running Quartex media Desktop. Enough to power a school.
I only have the weekends to work on Quartex Pascal, but I have spent the past 18 months tinkering away, making up for wasted time. So I’m just going to leave some pictures here for you to enjoy.
Note: I was asked on LinkedIn if this has anything to do with Smart Mobile Studio, and the answer is a resounding no. I have nothing to do with Smart any more. QTX Pascal is a completely separate project that is written from scratch by yours truly.
The QTX Framework was initially a library I created back in 2014, but it has later been completely overhauled and turned into a full RTL. It is not compatible with Smart Pascal and has a completely different architecture.
QTX Pascal is indirectly funded by the Amiga Retro Community (which might sound strange, but the technical level of that community is beyond anything I have encountered elsewhere) since QTX is central to the creation of the Quartex Media Desktop. It is a shame that Embarcadero decided to not back the project. The compiler and toolchain would have been a part of Delphi by now, and I wouldn’t have to write a separate IDE. But when they see what this system can deliver in terms of services, database work, mobile and embedded -they might regret it. The project only accepts donation funding, I am not interested in investors or partners. If you want a vision turned into reality, you gotta do it yourself. Everything else just gets in the way.
For developers by developers
Quartex Pascal is made for the community. It will be free for students and open-source projects. And a commercial license will never exceed $300. It is a shareware license and the financial aspects is purely to help fund further research and development of the desktop cloud platform. The final goal (CloudForge) is to compile the IDE itself to JavaScript, so people only need a browser to write enterprise level applications via Quartex Media Desktop. When that is finished, my work is done – and people have a clear path to the future.
Unlike other systems, QTX started with the non-visual stuff, so the system has a well implemented infrastructure for writing universal services and servers, using node.js as a deployment host. Services are also Docker friendly. Runs without change on Windows, Mac OS, Linux and a wealth of embedded systems and SBCs (single board computers)
A completely new RTL written from scratch generates close to native speed JS, highly compatible (even with legacy browsers) and rock solid
There are several display modes for QTX forms, from dynamic to absolute positioning. You can mix and match between HTML and QTX code, including a HTML5 compliant WYSIWYG editor and style manager. Makes content handling a lot easier
Write object pascal, JavaScript, HTML, LDEF (webassembly), node.js services – or mix and match between them all for maximum potential. Writing mobile applications is now ridiculously easy compared to “other tools” out there.
Oh and for the proverbial frosting — The full clustered Quartex Media desktop and services is a project type. Thats right. A complete cloud infrastructure suitable for teams, kiosks, embedded, schools, intranets – and even an replacement OS for ChromeOS. You don’t need to interface with Amazon, you get your own Amazon (optional naturally).
Filesystem over websocket, IPC between hosted apps and desktop, full back-end services that are clustered, and run on anything from a Raspberry PI 4 to low-cost ARM SBCs.
Web Assembly made easy. Both for Delphi and QTX
Let there be rock
Oh, and documentation. Loads and loads of documentation.
Proper documentation, both class overview and explanations that a human being has written is paramount for learning and getting up to speed quickly.
I don’t have vacation this year, which means I only have weekends to tinker away. But i have spent the past 18-ish months preparing and slowly finishing the pieces I needed. From vector containers to form design controls, to a completely re-written RTL from scratch — so yeah. This time I’m doing it my way.
C/C++ porting, QTX and general status
C is a language that I used to play around with a lot back in the Amiga days. I think the last time I used a C compiler to write a library must have been in 1992 or something like that? I held on to my Amiga 1200 for as long as i could – but having fallen completely in love with Pascal, I eventually switched to x86 and went down the Turbo Pascal road.
Lately however, C++ developers have been asking for their own Developer group on Facebook. I run several groups on Facebook in the so-called “developer” family. So you have Delphi Developer, FPC Developer, Node.JS Developer and now – C++Builder developer. The groups more or less tend to themselves, and the node.js and FPC groups are presently being seeded (meaning, that the member count is being grown for a period).
The C++Builder group however, is having the same activity level as the Delphi group almost, thanks to some really good developers that post links, tips and help solve questions. I was also fortunate enough to have David Millington come on the Admin team. David is leading the C++Builder project, so his insight and knowledge of both language and product is exemplary. Just like Jim McKeeth, he is a wonderful resource for the community and chime in with answers to tricky questions whenever he has time to spare.
Getting back in the saddle
Having working some 30 years with Pascal and Object Pascal, 25 of those years in Delphi, C/C++ is never far away. I have an article on the subject that i’ve written for the Idera Community website, so I wont dig too deep into that here — but needless to say, Rad Studio consists of two languages: Object Pascal and C/C++, so no matter how much you love either language, the other is never far away.
So I figured it was time for this old dog to learn some new tricks! I have always said that it’s wise to learn a language immediately below and above your comfort zone. So if Delphi is your favorite language, then C/C++ is below you (meaning: more low level and complex). Above you are languages like JavaScript and C#. Learning JavaScript makes strategic sense (or use DWScript to compile Pascal to JavaScript like I do).
When I started out, the immediate language below Object Pascal was never C, but assembler. So for the longest time I turned to assembler whenever I needed a speed boost; graphics manipulation and processing pixels is especially a field where assembly makes all the difference.
But since C++Builder is indeed an integral part of Rad Studio, and Object Pascal and C/C++ so intimately connected (they have evolved side by side), why not enjoy both assembly and C right?
So I decided to jump back into the saddle and see what I could make of it.
C/C++ is not as hard as you think
I’m having a ball writing C/C++, and just like Delphi – you can start where you are.
While I’m not going to rehash the article I have already prepared for the Idera Community pages here, I do want to encourage people to give it a proper try. I have always said that if you know an archetypal language, you can easily pick up other languages, because the archetypal languages will benefit you for a lifetime. This has to do with archetypal languages operating according to how computers really work; as opposed to optimistic languages (a term from the DB work, optimistic locking), also called contextual languages, like C#, Java, JavaScript etc. are based on how human beings would like things to be.
So I now had a chance to put my money where my mouth is.
When I left C back in the early 90s, I never bothered with OOP. I mean, I used C purely for shared libraries anyways, while the actual programs were done in Pascal or a hybrid language called Blitz Basic. The latter compiled to razor sharp machine code, and you could use inline assembly – which I used a lot back then (very few programmers on those machines went without assembler, it was almost given that you could use 68k in some capacity).
Without ruining the article about to be published, I had a great time with C++Builder. It took a few hours to get my bearings, but since both the VCL and FMX frameworks are there – you can approach C/C++ just like you would Object Pascal. So it’s a matter of getting an overview really.
Needless to say, I’ll be porting a fair share of my libraries to C/C++ when I have time (those that makes sense under that paradigme). It’s always good to push yourself and there are plenty of subtle differences that I found useful.
Quartex Media Desktop
When I last wrote about QTX we were nearing the completion of the FileSystem and Task Management service. The prototype had all its file-handling directly in the core service (or server) which worked just fine — but it was linked to the Smart Pascal RTL. It has taken time to write a new RTL + a full multi-user, platform independent service stack and desktop (phew!) but we are seeing progress!
The QTX Baseline backend services is now largely done
The filesystem service is now largely done! There are a few synchronous calls I want to get rid of, but thankfully my framework has both async and sync variations of all file procedures – so that is now finished.
To make that clearer: first I have to wrap and implement the functionality for the RTL. Once they are in the RTL, I can use those functions to build the service functions. So yeah, it’s been extremely elaborate — but thankfully it’s also become a rich, well organized codebase (both the RTL and the Quartex Media Desktop codebases) – so I think we are ready to get cracking on the core!
The core is still operating with the older API. So our next step is to remove that from the core and instead delegate calls to the filesystem to our new service. So the core will simply be reduced to a post-office or traffic officer if you like. Messages come in from the desktops, and the core delegates the messages to whatever service is in charge of them.
But, this also means that both the core and the desktop must use the new and fancy messages. And this is where I did something very clever.
While I was writing the service, I also write a client class to test (obviously). And the way the core works — means that the same client that the core use to talk to the services — can be used by the desktop as well.
So our work in the desktop to get file-access and drives running again, is to wrap the client in our TQTXDevice ancestor class. The desktop NEVER accesses the API directly. All it knows about are these device drivers (or object instances). Which is how we solve things like DropBox and Google Drive support. The desktop wont have the faintest clue that its using Dropbox, or copying files between a local disk and Google Drive for example — because it only communicates with these device classes.
Recursive stuff
One thing that sucked about node.js function for deleting a folder, is that it’s recursive parameter doesn’t work on Windows or OS X. So I had to implement a full recursive deletefolder routine manually. Not a big thing, but slightly more painful than expected under asynchronous execution. Thankfully, Object Pascal allows for inline defined procedures, so I didn’t have to isolate it in a separate class.
Here is some of the code, a tiny spec compared to the full shabam, but it gives you an idea of what life is like under async conditions:
unit service.file.core;
interface
{.$DEFINE DEBUG}
const
CNT_PREFS_DEFAULTPORT = 1883;
CNT_PREFS_FILENAME = 'QTXTaskManager.preferences.ini';
CNT_PREFS_DBNAME = 'taskdata.db';
CNT_ZCONFIG_SERVICE_NAME = 'TaskManager';
uses
qtx.sysutils,
qtx.json,
qtx.db,
qtx.logfile,
qtx.orm,
qtx.time,
qtx.node.os,
qtx.node.sqlite3,
qtx.node.zconfig,
qtx.node.cluster,
qtx.node.core,
qtx.node.filesystem,
qtx.node.filewalker,
qtx.fileapi.core,
qtx.network.service,
qtx.network.udp,
qtx.inifile,
qtx.node.inifile,
NodeJS.child_process,
ragnarok.types,
ragnarok.Server,
ragnarok.messages.base,
ragnarok.messages.factory,
ragnarok.messages.network,
service.base,
service.dispatcher,
service.file.messages;
type
TQTXTaskServiceFactory = class(TMessageFactory)
protected
procedure RegisterIntrinsic; override;
end;
TQTXFileWriteCB = procedure (TagValue: variant; Error: Exception);
TQTXFileStateCB = procedure (TagValue: variant; Error: Exception);
TQTXUnRegisterLocalDeviceCB = procedure (TagValue: variant; DiskName: string; Error: Exception);
TQTXRegisterLocalDeviceCB = procedure (TagValue: variant; LocalPath: string; Error: Exception);
TQTXFindDeviceCB = procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception);
TQTXGetDisksCB = procedure (TagValue: variant; Devices: JDeviceList; Error: Exception);
TQTXGetFileInfoCB = procedure (TagValue: variant; LocalName: string; Info: JStats; Error: Exception);
TQTXGetTranslatePathCB = procedure (TagValue: variant; Original, Translated: string; Error: Exception);
TQTXCheckDevicePathCB = procedure (TagValue: variant; PathName: string; Error: Exception);
TQTXServerExecuteCB = procedure (TagValue: variant; Data: string; Error: Exception);
TQTXTaskService = class(TRagnarokService)
private
FPrefs: TQTXIniFile;
FLog: TQTXLogEmitter;
FDatabase: TSQLite3Database;
FZConfig: TQTXZConfigClient;
FRegHandle: TQTXDispatchHandle;
FRegCount: integer;
procedure HandleGetDevices(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleGetDeviceByName(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleCreateLocalDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleDestroyDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleFileRead(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleFileReadPartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleGetFileInfo(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleFileDelete(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleFileWrite(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleFileWritePartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleFileRename(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleGetDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleMkDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure HandleRmDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
procedure ExecuteExternalJS(Params: array of string;
TagValue: variant; const CB: TQTXServerExecuteCB);
procedure SendError(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage; Message: string);
protected
function GetFactory: TMessageFactory; override;
procedure SetupPreferences(const CB: TRagnarokServiceCB);
procedure SetupLogfile(LogFileName: string;const CB: TRagnarokServiceCB);
procedure SetupDatabase(const CB: TRagnarokServiceCB);
procedure ValidateLocalDiskName(TagValue: variant; Username, DeviceName: string; CB: TQTXCheckDevicePathCB);
procedure RegisterLocalDevice(TagValue: variant; Username, DiskName: string; CB: TQTXRegisterLocalDeviceCB);
procedure UnRegisterLocalDevice(TagValue: variant; UserName, DiskName:string; CB: TQTXUnRegisterLocalDeviceCB);
procedure GetDevicesForUser(TagValue: variant; UserName: string; CB: TQTXGetDisksCB);
procedure FindDeviceByName(TagValue: variant; UserName, DiskName: string; CB: TQTXFindDeviceCB);
procedure FindDeviceByType(TagValue: variant; UserName: string; &Type: JDeviceType; CB: TQTXGetDisksCB);
procedure GetTranslatedPathFor(TagValue: variant; Username, FullPath: string; CB: TQTXGetTranslatePathCB);
procedure GetFileInfo(TagValue: variant; UserName: string; FullPath: string; CB: TQTXGetFileInfoCB);
procedure SetupTaskTable(const TagValue: variant; const CB: TRagnarokServiceCB);
procedure SetupOperationsTable(const TagValue: variant; const CB: TRagnarokServiceCB);
procedure SetupDeviceTable(const TagValue: variant; const CB: TRagnarokServiceCB);
procedure AfterServerStarted; override;
procedure BeforeServerStopped; override;
procedure Dispatch(Socket: TNJWebSocketSocket; Message: TQTXBaseMessage); override;
public
property Preferences: TQTXIniFile read FPrefs;
property Database: TSQLite3Database read FDatabase;
procedure SetupService(const CB: TRagnarokServiceCB);
constructor Create; override;
destructor Destroy; override;
end;
implementation
//#############################################################################
// TQTXFileenticationFactory
//#############################################################################
procedure TQTXTaskServiceFactory.RegisterIntrinsic;
begin
writeln("Registering task interface");
&Register(TQTXFileGetDeviceListRequest);
&Register(TQTXFileGetDeviceByNameRequest);
&Register(TQTXFileCreateLocalDeviceRequest);
&Register(TQTXFileDestroyDeviceRequest);
&Register(TQTXFileReadPartialRequest);
&Register(TQTXFileReadRequest);
&Register(TQTXFileWritePartialRequest);
&Register(TQTXFileWriteRequest);
&Register(TQTXFileDeleteRequest);
&Register(TQTXFileRenameRequest);
&Register(TQTXFileInfoRequest);
&Register(TQTXFileDirRequest);
&Register(TQTXMkDirRequest);
&Register(TQTXRmDirRequest);
&Register(TQTXFileRenameRequest);
&Register(TQTXFileDirRequest);
end;
//#############################################################################
// TQTXTaskService
//#############################################################################
constructor TQTXTaskService.Create;
begin
inherited Create;
FPrefs := TQTXIniFile.Create();
FLog := TQTXLogEmitter.Create();
FDatabase := TSQLite3Database.Create(nil);
FZConfig := TQTXZConfigClient.Create();
FZConfig.Port := 2292;
self.OnUserSignedOff := procedure (Sender: TObject; Username: string)
begin
WriteToLogF("We got a service signal! User [%s] has signed off completely", [Username]);
end;
MessageDispatch.RegisterMessage(TQTXFileGetDeviceListRequest, @HandleGetDevices);
MessageDispatch.RegisterMessage(TQTXFileGetDeviceByNameRequest, @HandleGetDeviceByName);
MessageDispatch.RegisterMessage(TQTXFileCreateLocalDeviceRequest, @HandleCreateLocalDevice);
MessageDispatch.RegisterMessage(TQTXFileDestroyDeviceRequest, @HandleDestroyDevice);
MessageDispatch.RegisterMessage(TQTXFileReadRequest, @HandleFileRead);
MessageDispatch.RegisterMessage(TQTXFileReadPartialRequest, @HandleFileReadPartial);
MessageDispatch.RegisterMessage(TQTXFileWriteRequest, @HandleFileWrite);
MessageDispatch.RegisterMessage(TQTXFileWritePartialRequest, @HandleFileWritePartial);
MessageDispatch.RegisterMessage(TQTXFileInfoRequest, @HandleGetFileInfo);
MessageDispatch.RegisterMessage(TQTXFileDeleteRequest, @HandleFileDelete);
MessageDispatch.RegisterMessage(TQTXMkDirRequest, @HandleMkDir);
MessageDispatch.RegisterMessage(TQTXRmDirRequest, @HandleRmDir);
MessageDispatch.RegisterMessage(TQTXFileRenameRequest, @HandleFileRename);
MessageDispatch.RegisterMessage(TQTXFileDirRequest, @HandleGetDir);
end;
destructor TQTXTaskService.Destroy;
begin
// decouple logger from our instance
self.logging := nil;
// Release prefs + log
FPrefs.free;
FLog.free;
FZConfig.free;
inherited;
end;
procedure TQTXTaskService.SendError(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage; Message: string);
begin
var reply := TQTXErrorMessage.Create(request.ticket);
try
reply.Code := CNT_MESSAGE_CODE_ERROR;
reply.Routing.TagValue := Request.Routing.TagValue;
reply.Response := Message;
if Socket.ReadyState = rsOpen then
begin
try
Socket.Send( reply.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end else
WriteToLog("Failed to dispatch error, socket is closed error");
finally
reply.free;
end;
end;
procedure TQTXTaskService.ExecuteExternalJS(Params: array of string;
TagValue: variant; const CB: TQTXServerExecuteCB);
begin
var LTask: JChildProcess;
var lOpts := TVariant.CreateObject();
lOpts.shell := false;
lOpts.detached := true;
Params.insert(0, '--no-warnings');
// Spawn a new process, this creates a new shell interface
try
LTask := child_process().spawn('node', Params, lOpts );
except
on e: exception do
begin
if assigned(CB) then
CB(TagValue, e.message, e);
exit;
end;
end;
// Map general errors on process level
LTask.on('error', procedure (error: variant)
begin
{$IFDEF DEBUG}
writeln("error->" + error.toString());
{$ENDIF}
WriteToLog(error.toString());
if assigned(CB) then
CB(TagValue, "", nil);
end);
// map stdout so we capture the output
LTask.stdout.on('data', procedure (data: variant)
begin
if assigned(CB) then
CB(TagValue, data.toString(), nil);
end);
// map stderr so we can capture exception messages
LTask.stderr.on('data', procedure (error:variant)
begin
{$IFDEF DEBUG}
writeln("stdErr->" + error.toString());
{$ENDIF}
if assigned(CB) then
CB(TagValue, "", nil);
WriteToLog(error.toString());
end);
end;
function TQTXTaskService.GetFactory: TMessageFactory;
begin
result := TQTXTaskServiceFactory.Create();
end;
procedure TQTXTaskService.SetupService(const CB: TRagnarokServiceCB);
begin
SetupPreferences( procedure (Error: Exception)
begin
// No logfile yet setup (!)
if Error nil then
begin
WriteToLog("Preferences setup: Failed!");
WriteToLog(error.message);
raise error;
end else
WriteToLog("Preferences setup: OK");
// logfile-name is always relative to the executable
var LLogFileName := TQTXNodeFileUtils.IncludeTrailingPathDelimiter( TQTXNodeFileUtils.GetCurrentDirectory );
LLogFileName += FPrefs.ReadString('log', 'logfile', 'log.txt');
// Port is defined in the ancestor, so we assigns it here
Port := FPrefs.ReadInteger('networking', 'port', CNT_PREFS_DEFAULTPORT);
SetupLogfile(LLogFileName, procedure (Error: Exception)
begin
if Error nil then
begin
WriteToLog("Logfile setup: Failed!");
WriteToLog(error.message);
raise error;
end else
WriteToLog("Logfile setup: OK");
SetupDatabase( procedure (Error: Exception)
begin
if Error nil then
begin
WriteToLog("Database setup: Failed!");
WriteToLog(error.message);
if assigned(CB) then
CB(Error)
else
raise Error;
end else
WriteToLog("Database setup: OK");
if assigned(CB) then
CB(nil);
end);
end);
end);
end;
procedure TQTXTaskService.SetupPreferences(const CB: TRagnarokServiceCB);
begin
var lBasePath := TQTXNodeFileUtils.GetCurrentDirectory;
var LPrefsFile := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + CNT_PREFS_FILENAME;
if TQTXNodeFileUtils.FileExists(LPrefsFile) then
begin
FPrefs.LoadFromFile(nil, LPrefsFile, procedure (TagValue: variant; Error: Exception)
begin
if Error nil then
begin
if assigned(CB) then
CB(Error)
else
raise Error;
exit;
end;
if assigned(CB) then
CB(nil);
end);
end else
begin
var LError := Exception.Create('Could not locate preferences file: ' + LPrefsFile);
WriteToLog(LError.message);
if assigned(CB) then
CB(LError)
else
raise LError;
end;
end;
procedure TQTXTaskService.SetupLogfile(LogFileName: string;const CB: TRagnarokServiceCB);
begin
// Attempt to open logfile
// Note: Log-object error options is set to throw exceptions
try
FLog.Open(LogFileName);
except
on e: exception do
begin
if assigned(CB) then
begin
CB(e);
exit;
end else
begin
writeln(e.message);
raise;
end;
end;
end;
// We inherit from TQTXLogObject, which means we can pipe
// all errors etc directly using built-in functions. So here
// we simply glue our instance to the log-file, and its all good
self.Logging := FLog as IQTXLogClient;
if assigned(CB) then
CB(nil);
end;
procedure TQTXTaskService.FindDeviceByType(TagValue: variant; UserName: string; &Type: JDeviceType; CB: TQTXGetDisksCB);
begin
UserName := username.trim().ToLower();
if Username.length < 1 then
begin
WriteToLog("Failed to lookup disk, username was invalid error");
var lError := EException.Create("Failed to lookup devices, invalid username");
if assigned(CB) then
CB(TagValue, nil, lError)
else
raise lError;
exit;
end;
GetDevicesForUser(TagValue, Username,
procedure (TagValue: variant; Devices: JDeviceList; Error: Exception)
begin
if Error nil then
begin
if assigned(CB) then
CB(TagValue, nil, Error)
else
raise Error;
exit;
end;
var x := 0;
while x < Devices.dlDrives.Count do
begin
if Devices.dlDrives[x].&Type &Type then
begin
Devices.dlDrives.delete(x, 1);
continue;
end;
inc(x);
end;
if assigned(CB) then
CB(TagValue, Devices, nil);
end);
end;
procedure TQTXTaskService.FindDeviceByName(TagValue: variant; Username, DiskName: string; CB: TQTXFindDeviceCB);
begin
UserName := username.trim().ToLower();
if Username.length < 1 then
begin
var lLogText := "Failed to lookup device, username was invalid error";
WriteToLog(lLogText);
var lError := EException.Create(lLogText);
if assigned(CB) then
CB(TagValue, nil, lError)
else
raise lError;
exit;
end;
DiskName := DiskName.trim();
if DiskName.length < 1 then
begin
var lLogText := "Failed to lookup device, disk-name was invalid error";
WriteToLog(lLogText);
var lError := EException.Create(lLogText);
if assigned(CB) then
CB(TagValue, nil, lError)
else
raise lError;
exit;
end;
GetDevicesForUser(TagValue, Username,
procedure (TagValue: variant; Devices: JDeviceList; Error: Exception)
begin
if Error nil then
begin
if assigned(CB) then
CB(TagValue, nil, Error)
else
raise Error;
exit;
end;
DiskName := DiskName.trim().ToLower();
var lDiskInfo: JDeviceInfo := nil;
for var disk in Devices.dlDrives do
begin
if disk.Name.ToLower() = DiskName then
begin
lDiskInfo := disk;
break;
end;
end;
if assigned(CB) then
CB(TagValue, lDiskInfo, nil);
end);
end;
procedure TQTXTaskService.GetTranslatedPathFor(TagValue: variant; Username, FullPath: string; CB: TQTXGetTranslatePathCB);
begin
var lParser := TQTXPathParser.Create();
try
var lInfo: TQTXPathData;
if lparser.Parse(FullPath, lInfo) then
begin
// Locate the device for the path belonging to the user
FindDeviceByName(TagValue, UserName, lInfo.MountPart,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
if Error nil then
begin
if assigned(CB) then
CB(TagValue, FullPath, '', Error)
else
raise Error;
exit;
end;
if Device.&Type dtLocal then
begin
var lError := EException.CreateFmt('Failed to translate path, device [%s] is not local error', [Device.Name]);
if assigned(CB) then
CB(TagValue, FullPath, '', Error)
else
raise Error;
exit;
end;
// We want the path + filename, so we can append that to
// the actual localized filesystem
var lExtract := FullPath;
delete(lExtract, 1, lInfo.MountPart.Length + 1);
// Construct complete storage location
var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;
// Return translated path
if assigned(CB) then
CB(TagValue, FullPath, lFullPath, nil);
end);
end else
begin
var lErr := EException.CreateFmt("Invalid path [%s] error", [FullPath]);
if assigned(CB) then
CB(TagValue, FullPath, '', lErr)
else
raise lErr;
end;
finally
lParser.free;
end;
end;
procedure TQTXTaskService.GetFileInfo(TagValue: variant; UserName, FullPath: string; CB: TQTXGetFileInfoCB);
begin
var lParser := TQTXPathParser.Create();
try
var lInfo: TQTXPathData;
if lparser.Parse(FullPath, lInfo) then
begin
// Locate the device for the path belonging to the user
FindDeviceByName(TagValue, UserName, lInfo.MountPart,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
if Error nil then
begin
if assigned(CB) then
CB(TagValue, '', nil, Error)
else
raise Error;
exit;
end;
case Device.&Type of
dtLocal:
begin
// We want the path + filename, so we can append that to
// the actual localized filesystem
var lExtract := FullPath;
delete(lExtract, 1, lInfo.MountPart.Length + 1);
// Construct complete storage location
var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;
// Call the underlying OS to get the file statistics
NodeJsFsAPI().lStat(lFullPath,
procedure (Error: JError; Stats: JStats)
begin
if Error nil then
begin
var lError := EException.Create(Error.message);
if assigned(CB) then
CB(TagValue, lFullPath, nil, lError)
else
raise lError;
exit;
end;
// And deliver
if assigned(CB) then
CB(TagValue, lFullPath, Stats, nil);
end);
end;
dtDropbox, dtGoogle, dtMsDrive:
begin
var lError := EException.Create("Cloud bindings not activated error");
if assigned(CB) then
CB(TagValue, '', nil, lError)
end;
end;
end);
end else
begin
var lErr := EException.CreateFmt("Invalid path [%s] error", [FullPath]);
if assigned(CB) then
CB(TagValue, '', nil, lErr)
else
raise lErr;
end;
finally
lParser.free;
end;
end;
procedure TQTXTaskService.GetDevicesForUser(TagValue: variant; Username: string; CB: TQTXGetDisksCB);
begin
UserName := username.trim().ToLower();
if Username.length < 1 then
begin
WriteToLog("Failed to lookup devices, username was invalid error");
var lError := EException.Create("Failed to lookup devices, invalid username");
if assigned(CB) then
CB(TagValue, nil, lError)
else
raise lError;
exit;
end;
var lTransaction: TQTXReadTransaction;
if not TSQLite3Database(DataBase).CreateReadTransaction(lTransaction) then
begin
var lErr := EException.Create("Failed to create read-transaction error");
if assigned(cb) then
CB(TagValue, nil, lErr)
else
raise lErr;
exit;
end;
var lQuery := TSQLite3ReadTransaction(lTransaction);
lQuery.SQL := "select * from devices where owner=?";
lQuery.Parameters.AddValueOnly(Username);
lQuery.Execute(
procedure (Sender: TObject; Error: Exception)
begin
if Error nil then
begin
if assigned(CB) then
CB(TagValue, nil, Error)
else
raise Error;
exit;
end;
var lDisks := new JDeviceList();
lDisks.dlUser := UserName;
for var x := 0 to lQuery.datarows.length-1 do
begin
var lInfo := new JDeviceInfo();
lInfo.Name := lQuery.datarows[x]["name"];
lInfo.&Type := JDeviceType( lQuery.datarows[x]["type"] );
lInfo.owner := lQuery.datarows[x]["owner"];
lInfo.location := lQuery.datarows[x]["location"];
lInfo.APIKey := lQuery.datarows[x]["apikey"];
lInfo.APISecret := lQuery.datarows[x]["apisecret"];
lInfo.APIPassword := lQuery.datarows[x]["apipassword"];
lInfo.APIUser := lQuery.datarows[x]["apiuser"];
lDisks.dlDrives.add(lInfo);
end;
try
if assigned(CB) then
CB(TagValue, lDisks, nil);
finally
lQuery.free;
end;
end);
end;
procedure TQTXTaskService.ValidateLocalDiskName(TagValue: variant; Username, DeviceName: string; CB: TQTXCheckDevicePathCB);
begin
var Filename := 'disk.' + username + '.' + DeviceName + '.' + ord(JDeviceType.dtLocal).ToString();
var LBasePath := TQTXNodeFileUtils.GetCurrentDirectory();
LBasePath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + 'userdevices';
// Make sure the device folder is there
if not TQTXNodeFileUtils.DirectoryExists(LBasePath) then
begin
var lError := EException.CreateFmt("Directory not found: %s", [lBasePath]);
if assigned(CB) then
CB(TagValue, '', lError)
else
raise lError;
exit;
end;
lBasePath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + Filename;
if TQTXNodeFileUtils.DirectoryExists(LBasePath) then
begin
var lError := EException.CreateFmt("Path already exist error [%s]", [lBasePath]);
if assigned(CB) then
CB(TagValue, '', lError)
else
raise lError;
exit;
end;
// OK, folder is not created yet, so its good to go
if assigned(CB) then
CB(TagValue, Filename, nil);
end;
procedure TQTXTaskService.UnRegisterLocalDevice(TagValue: variant; UserName, DiskName: string; CB: TQTXUnRegisterLocalDeviceCB);
begin
WriteToLogF("Removing local device [%s] for user [%s] ", [DiskName, Username]);
// Check username string
UserName := username.trim().ToLower();
if Username.length < 1 then
begin
WriteToLog("Failed to unregister device, username was invalid error");
var lError := EException.Create("Failed to register device, invalid username");
if assigned(CB) then
CB(TagValue, DiskName, lError)
else
raise lError;
exit;
end;
// Check diskname string
DiskName := DiskName.trim().ToLower();
if DiskName.length < 1 then
begin
WriteToLog("Failed to unregister device, disk-name was invalid error");
var lError := EException.Create("Failed to register device, invalid disk-name");
if assigned(CB) then
CB(TagValue, DiskName, lError)
else
raise lError;
exit;
end;
FindDeviceByName(TagValue, Username, DiskName,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
// Did the search fail?
if Error nil then
begin
WriteToLog(Error.message);
if assigned(CB) then
CB(TagValue, DiskName, Error)
else
raise Error;
exit;
end;
// Make sure the device is local
if Device.&Type dtLocal then
begin
var lError := EException.CreateFmt('Failed to translate path, device [%s] is not local error', [Device.Name]);
if assigned(CB) then
CB(TagValue, DiskName, Error)
else
raise Error;
exit;
end;
// Delete record from database
var lWriter: TQTXWriteTransaction;
if FDatabase.CreateWriteTransaction(lWriter) then
begin
lWriter.SQL := "delete from profiles where user = ? and name = ?;";
lWriter.Parameters.AddValueOnly(Username);
lWriter.Parameters.AddValueOnly(DiskName);
lWriter.Execute(
procedure (Sender: TObject; Error: Exception)
begin
try
if Error nil then
begin
if assigned(CB) then
CB(TagValue, DiskName, Error)
else
raise Error;
exit;
end;
// Construct complete storage location
var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
// Now delete the disk-drive directory
TQTXNodeFileUtils.DeleteDirectory(nil, lFullPath,
procedure (TagValue: variant; Path: string; Error: Exception)
begin
if assigned(CB) then
CB(TagValue, DiskName, Error)
end);
finally
lWriter.free;
lWriter := nil;
end;
end);
end;
end);
end;
procedure TQTXTaskService.RegisterLocalDevice(TagValue: variant; Username, DiskName: string; CB: TQTXRegisterLocalDeviceCB);
begin
WriteToLogF("Adding local device [%s] for user [%s] ", [DiskName, Username]);
UserName := username.trim().ToLower();
if Username.length < 1 then
begin
WriteToLog("Failed to register device, username was invalid error");
var lError := EException.Create("Failed to register device, invalid username");
if assigned(CB) then
CB(TagValue, '', lError)
else
raise lError;
exit;
end;
DiskName := DiskName.trim().ToLower();
if DiskName.length < 1 then
begin
WriteToLog("Failed to register device, disk-name was invalid error");
var lError := EException.Create("Failed to register device, invalid disk-name");
if assigned(CB) then
CB(TagValue, '', lError)
else
raise lError;
exit;
end;
FindDeviceByName(TagValue, Username, DiskName,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
// Did the search fail?
if Error nil then
begin
WriteToLog(Error.message);
if assigned(CB) then
CB(TagValue, '', Error)
else
raise Error;
exit;
end;
// Does a device that match already exist?
if Device nil then
begin
var lError := EException.CreateFmt("Failed to create device [%s], device already exists", [DiskName]);
if assigned(CB) then
CB(TagValue, '', lError)
else
raise lError;
exit;
end;
// make sure the device-folder does not exist, so we can create it
ValidateLocalDiskName(TagValue, Username, DiskName,
procedure (TagValue: variant; PathName: string; Error: Exception)
begin
if Error nil then
begin
if assigned(CB) then
CB(TagValue, '', Error)
else
raise Error;
exit;
end;
// ValidateLocalDiskName only returns the valid directory-name,
// not a full path -- so we need to build up the full targetpath
var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + PathName;
TQTXNodeFileUtils.CreateDirectory(nil, lFullPath,
procedure (TagValue: variant; Path: string; Error: exception)
begin
if Error nil then
begin
var lError := EException.CreateFmt("Failed to create device [%s] with path: %s", [DiskName, lFullPath]);
if assigned(CB) then
CB(TagValue, PathName, lError)
else
raise lError;
exit;
end;
FDatabase.Execute(
#'insert into devices (type, owner, name, location)
values(?, ?, ?, ?);',
[ord(JDeviceType.dtLocal), UserName, Diskname, PathName] ,
procedure (Sender: TObject; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
if assigned(CB) then
CB(TagValue, PathName, Error)
else
raise Error;
exit;
end;
WriteToLogF("Device [%s] added to database user [%s]", [DiskName, UserName]);
if assigned(CB) then
CB(TagValue, PathName, nil);
end);
end);
end);
end);
end;
procedure TQTXTaskService.SetupDeviceTable(const TagValue: variant; const CB: TRagnarokServiceCB);
begin
FDatabase.Execute(
#'
create table if not exists devices
(
id integer primary key AUTOINCREMENT,
type integer,
owner text,
name text,
location text,
apikey text,
apisecret text,
apipassword text,
apiuser text
);
', [],
procedure (Sender: TObject; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
if assigned(CB) then
CB(Error)
else
raise Error;
exit;
end else
if assigned(CB) then
CB(nil);
end);
end;
procedure TQTXTaskService.SetupTaskTable(const TagValue: variant; const CB: TRagnarokServiceCB);
begin
FDatabase.Execute(
#'
create table if not exists tasks
(
id integer primary key AUTOINCREMENT,
state integer,
username text,
created real
);
', [],
procedure (Sender: TObject; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
if assigned(CB) then
CB(Error)
else
raise Error;
exit;
end else
if assigned(CB) then
CB(nil);
end);
end;
procedure TQTXTaskService.SetupOperationsTable(const TagValue: variant; const CB: TRagnarokServiceCB);
begin
FDatabase.Execute(
#'
create table if not exists operations
(
id integer primary key AUTOINCREMENT,
username text,
taskid integer,
name text,
filename text
);
', [],
procedure (Sender: TObject; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
if assigned(CB) then
CB(Error)
else
raise Error;
exit;
end else
if assigned(CB) then
CB(nil);
end);
end;
procedure TQTXTaskService.SetupDatabase(const CB: TRagnarokServiceCB);
begin
// Try to read database-path from preferences file
var LDbFileToOpen := FPrefs.ReadString("database", "database_name", "");
// Trim away spaces, check if there is a filename
LDbFileToOpen := LDbFileToOpen.trim();
if LDbFileToOpen.length < 1 then
begin
// No filename? Fall back on pre-defined file in CWD
var LBasePath := TQTXNodeFileUtils.GetCurrentDirectory();
LDbFileToOpen := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(LBasePath) + CNT_PREFS_DBNAME;
end;
FDatabase.AccessMode := TSQLite3AccessMode.sqaReadWriteCreate;
FDatabase.Open(LDbFileToOpen,
procedure (Sender: TObject; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
if assigned(CB) then
CB(Error)
else
raise Error;
exit;
end;
WriteToLog("Initializing task table");
SetupTaskTable(nil, procedure (Error: exception)
begin
if Error nil then
begin
WriteToLog("Tasks initialized: **failed");
WriteToLog(error.message);
if assigned(CB) then
CB(Error)
else
raise Error;
exit;
end else
writeToLog("Tasks initialized: OK");
WriteToLog("Initializing operations table");
SetupOperationsTable(nil, procedure (Error: exception)
begin
if Error nil then
begin
WriteToLog("Operations initialized: **failed");
WriteToLog(error.message);
if assigned(CB) then
CB(Error);
exit;
end else
writeToLog("Operations initialized: OK");
WriteToLog("Initializing device table");
SetupDeviceTable(nil, procedure (Error: exception)
begin
if Error nil then
begin
WriteToLog("Device-table initialized: **failed");
WriteToLog(error.message);
if assigned(CB) then
CB(Error);
exit;
end else
writeToLog("Device-table initialized: OK");
if assigned(CB) then
CB(nil);
end);
end);
end);
end);
end;
procedure TQTXTaskService.HandleFileRead(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileReadRequest(request);
var lUserName := lRequest.UserName;
var lFileName := lRequest.FileName;
// Check filename length
if lFileName.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
GetFileInfo(lRequest, lUserName, lFileName,
procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
var lOptions: TReadFileOptions;
lOptions.encoding := 'binary';
NodeJsFsAPI().readFile(LocalFile, lOptions,
procedure (Error: JError; Data: JNodeBuffer)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
var lResponse := TQTXFileReadResponse.Create(Request.Ticket);
lResponse.UserName := lUserName;
lResponse.Routing.TagValue := request.routing.tagValue;
lResponse.FileName := lFileName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
// Convert filedata in one pass
try
var lConvert := TDataTypeConverter.Create();
try
lResponse.Attachment.AppendBytes( lConvert.TypedArrayToBytes(Data) );
finally
lConvert.free;
end;
except
on e: exception do
begin
WriteToLog(e.message);
SendError(Socket, Request, e.Message);
exit;
end;
end;
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end);
end;
procedure TQTXTaskService.HandleFileReadPartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileReadPartialRequest(request);
var lUserName := lRequest.UserName;
var lFileName := lRequest.FileName;
var lStart := lRequest.Offset;
var lSize := lRequest.Size;
// Check filename length
if lFileName.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
if lSize < 1 then
begin
SendError(Socket, Request, "Read failed, invalid size error");
exit;
end;
if lStart < 0 then
begin
SendError(Socket, Request, "Read failed, invalid offset error");
exit;
end;
GetFileInfo(lRequest, lUserName, lFileName,
procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
if lStart > Info.size then
begin
SendError(Socket, Request, "Read failed, offset beyond filesize error");
exit;
end;
NodeJsFsAPI().open(LocalFile, "r",
procedure (Error: JError; Fd: THandle)
begin
if error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
var Data = new JNodeBuffer(lSize);
NodeJsFsAPI().read(Fd, Data, 0, lSize, lStart,
procedure (Error: JError; BytesRead: integer; buffer: JNodeBuffer)
begin
if Error nil then
begin
NodeJsFsAPI().closeSync(Fd);
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
// Close the file-handle and return data
NodeJsFsAPI().close(Fd, procedure (Error: JError)
begin
var lResponse := TQTXFileReadPartialResponse.Create(Request.Ticket);
lResponse.UserName := lUserName;
lResponse.Routing.TagValue := request.routing.tagValue;
lResponse.FileName := lFileName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
// Only encode data if read
if BytesRead > 0 then
begin
// Convert filedata in one pass
try
var lConvert := TDataTypeConverter.Create();
try
lResponse.Attachment.AppendBytes( lConvert.TypedArrayToBytes(buffer) );
finally
lConvert.free;
end;
except
on e: exception do
begin
WriteToLog(e.message);
SendError(Socket, Request, e.Message);
exit;
end;
end;
end;
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end);
end);
end);
end;
procedure TQTXTaskService.HandleFileWrite(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileWriteRequest(request);
var lFileName := lRequest.FileName.trim();
var lUserName := lRequest.UserName.trim();
var FullPath := lFileName;
// Check filename length
if lFileName.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
var lParser := TQTXPathParser.Create();
try
var lInfo: TQTXPathData;
if lparser.Parse(FullPath, lInfo) then
begin
// Locate the device for the path belonging to the user
FindDeviceByName(nil, lUserName, lInfo.MountPart,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
case Device.&Type of
dtLocal:
begin
// We want the path + filename, so we can append that to
// the actual localized filesystem
var lExtract := FullPath;
delete(lExtract, 1, lInfo.MountPart.Length + 1);
// Construct complete storage location
var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;
// Extract data to be appended, if any
// note: null bytes should be allowed, it should just create the file
var lBytes: array of UInt8;
if lRequest.attachment.Size > 0 then
lBytes := lRequest.Attachment.ToBytes();
// Write the data to the file
NodeJsFsAPI().writeFile(lFullPath, lBytes,
procedure (Error: JError)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
// Setup response object
var lResponse := TQTXFileWriteResponse.Create(lRequest.Ticket);
lResponse.UserName := lUserName;
lResponse.FileName := lFileName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
// Send success response
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end;
dtDropbox, dtGoogle, dtMsDrive:
begin
var lErrorText := Format("Clound bindings not active error [%s]", [lRequest.FileName]);
WriteToLog(lErrorText);
SendError(Socket, Request, lErrorText);
end;
end;
end);
end else
begin
SendError(Socket, Request, format("Invalid path [%s] error", [FullPath]));
end;
finally
lParser.free;
end;
end;
procedure TQTXTaskService.HandleFileWritePartial(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileWritePartialRequest(request);
var lFileName := lRequest.FileName.trim();
var lUserName := lRequest.UserName.trim();
var lFileOffset := lRequest.Offset;
// Check filename length
if lFileName.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
var FullPath := lFileName;
var lParser := TQTXPathParser.Create();
try
var lInfo: TQTXPathData;
if lparser.Parse(FullPath, lInfo) then
begin
// Locate the device for the path belonging to the user
FindDeviceByName(nil, lUserName, lInfo.MountPart,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
case Device.&Type of
dtLocal:
begin
// We want the path + filename, so we can append that to
// the actual localized filesystem
var lExtract := FullPath;
delete(lExtract, 1, lInfo.MountPart.Length + 1);
// Construct complete storage location
var lFullPath := TQTXNodeFileUtils.GetCurrentDirectory();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + 'userdevices';
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + Device.location.trim();
lFullPath := TQTXNodeFileUtils.IncludeTrailingPathDelimiter(lFullPath) + lExtract;
// Extract data to be appended, if any
// note: null bytes should be allowed, it should just create the file
var lBytes: array of UInt8;
if lRequest.attachment.Size > 0 then
lBytes := lRequest.Attachment.ToBytes();
var lAccess := TQTXNodeFile.Create();
lAccess.Open(lFullPath, TQTXNodeFileMode.nfWrite,
procedure (Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
lAccess.Write(lBytes, lFileOffset,
procedure (Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
// Setup response object
var lResponse := TQTXFileWriteResponse.Create(lRequest.Ticket);
lResponse.UserName := lUserName;
lResponse.FileName := lFileName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
// Send success response
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end);
end;
dtDropbox, dtGoogle, dtMsDrive:
begin
var lErrorText := Format("Clound bindings not active error [%s]", [lRequest.FileName]);
WriteToLog(lErrorText);
SendError(Socket, Request, lErrorText);
end;
end;
end);
end else
begin
SendError(Socket, Request, format("Invalid path [%s] error", [FullPath]));
end;
finally
lParser.free;
end;
end;
procedure TQTXTaskService.HandleRmDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXRmDirRequest(request);
var lUserName := lRequest.UserName.trim();
var lDirPath := lRequest.DirPath.trim();
if lDirPath.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lDirPath) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
var lParser := TQTXPathParser.Create();
try
var lInfo: TQTXPathData;
if lParser.Parse(lDirPath, lInfo) then
begin
GetTranslatedPathFor(nil, lUserName, lDirPath,
procedure (TagValue: variant; Original, Translated: string; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
if not TQTXNodeFileUtils.DirectoryExists(Translated) then
begin
WriteToLogF("RmDir Failed, directory [%s] does not exist", [Translated]);
SendError(Socket, Request, Format("RmDir failed, directory [%s] does not exist", [Original]));
exit;
end;
TQTXNodeFileUtils.DeleteDirectory(nil, Translated,
procedure (TagValue: variant; Path: string; Error: Exception)
begin
if error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
// Setup response object
var lResponse := TQTXRmDirResponse.Create(lRequest.Ticket);
lResponse.UserName := lUserName;
lResponse.DirPath := lDirPath;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
lResponse.Routing.TagValue := lRequest.Routing.TagValue;
// Send success response
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end);
end else
begin
var lText := format("RmDir failed, invalid path [%s] error", [lDirPath]);
WriteToLog(lText);
SendError(Socket, Request, lText);
end;
finally
lParser.free;
end;
end;
procedure TQTXTaskService.HandleMkDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXMkDirRequest(request);
var lUserName := lRequest.UserName.trim();
var lDirPath := lRequest.DirPath.trim();
if lDirPath.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lDirPath) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
var lParser := TQTXPathParser.Create();
try
var lInfo: TQTXPathData;
if lparser.Parse(lDirPath, lInfo) then
begin
GetTranslatedPathFor(nil, lUserName, lDirPath,
procedure (TagValue: variant; Original, Translated: string; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
TQTXNodeFileUtils.DirectoryExists(nil, Translated,
procedure (TagValue: variant; Path: string; Error: Exception)
begin
if Error nil then
begin
WriteToLogF("MkDir Failed, directory [%s] already exists", [Translated]);
SendError(Socket, Request, Format("MkDir Failed, directory [%s] already exists", [Original]));
exit;
end;
TQTXNodeFileUtils.CreateDirectory(nil, Translated,
procedure (TagValue: variant; Path: string; Error: Exception)
begin
if Error nil then
begin
WriteToLogF("MkDir Failed, directory [%s] could not be created", [Original]);
SendError(Socket, Request, Format("MkDir Failed, directory [%s] could not be created", [Translated]));
exit;
end;
// Setup response object
var lResponse := TQTXMkDirResponse.Create(lRequest.Ticket);
lResponse.UserName := lUserName;
lResponse.DirPath := lDirPath;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
lResponse.Routing.TagValue := lRequest.Routing.TagValue;
// Send success response
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end);
end);
end else
begin
var lText := format("MkDir Failed, invalid path [%s] error", [lDirPath]);
WriteToLog(lText);
SendError(Socket, Request, lText);
end;
finally
lParser.free;
end;
end;
procedure TQTXTaskService.HandleFileDelete(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileDeleteRequest(Request);
var lUserName := lRequest.UserName.trim();
var lFileName := lRequest.FileName.trim();
if lFileName.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
GetFileInfo(lRequest, lUserName, lFileName,
procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
if not Info.isFile then
begin
SendError(Socket, Request, "Filesystem object is not a file error");
exit;
end;
NodeJsFsAPI().unlink(LocalFile,
procedure (Error: JError)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.message);
exit;
end;
var lResponse := new TQTXFileDeleteResponse(lRequest.Ticket);
lResponse.Routing.TagValue := request.Routing.TagValue;
lResponse.UserName := lUserName;
lResponse.FileName := lFileName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end);
end;
procedure TQTXTaskService.HandleFileRename(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileRenameRequest(Request);
var lUserName := lRequest.UserName.trim();
var lFileName := lRequest.FileName.trim();
var lNewName := lRequest.NewName.trim();
// Check filename length
if lFileName.length < 1 then
begin
SendError(Socket, Request, Format("Invalid or empty from-filename [%s] error", [lFileName]) );
exit;
end;
// check newname length
if lNewName.length 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
if pos(lTemp, lNewName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
if pos(lTemp, lNewName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
GetFileInfo(lRequest, lUserName, lFileName,
procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
if not Info.isFile then
begin
SendError(Socket, Request, "Filesystem object is not a file error");
exit;
end;
GetTranslatedPathFor(nil, lUsername, lNewName,
procedure (TagValue: variant; Original, Translated: string; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
NodeJsFsAPI().rename(LocalFile, Translated,
procedure (Error: JError)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.message);
exit;
end;
var lResponse := new TQTXFileRenameResponse(lRequest.Ticket);
lResponse.Routing.TagValue := request.Routing.TagValue;
lResponse.UserName := lUserName;
lResponse.FileName := lFileName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end);
end);
end;
procedure TQTXTaskService.HandleGetDir(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileDirRequest(Request);
var lUserName := lRequest.UserName.trim();
var lPath := lRequest.Path.trim();
// prevent path escape attempts
var lTemp := "../";
if pos(lTemp, lPath) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lPath) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
GetTranslatedPathFor(nil, lUserName, lPath,
procedure (TagValue: variant; Original, Translated: string; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
//writeln("Translated path is:" + Translated);
if not TQTXNodeFileUtils.DirectoryExists(Translated) then
begin
WriteToLogF("GetDir Failed, directory [%s] does not exist", [Translated]);
SendError(Socket, Request, Format("GetDir failed, directory [%s] does not exist", [Original]));
exit;
end;
var lWalker := TQTXFileWalker.Create();
lWalker.Examine(Translated, procedure (Sender: TQTXFileWalker; Error: EException)
begin
if Error nil then
begin
WriteToLogF("GetDir Failed: %s", [Error.Message]);
SendError(Socket, Request, Format("GetDir failed: %s", [Error.Message]));
exit;
end;
// Get the directory data, swap out the path
// record with the original [amiga] style path
var lData := Sender.ExtractList();
lData.dlPath := Original;
var lResponse := new TQTXFileDirResponse(lRequest.Ticket);
lResponse.Routing.TagValue := request.Routing.TagValue;
lResponse.UserName := lUserName;
lResponse.Path := lPath;
lResponse.Assign( lData );
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
// release instance in 100ms
TQTXDispatch.execute(procedure ()
begin
try
lWalker.free
except
on e: exception do
begin
WriteToLogF("Failed to release file-walker instance: %s", [e.message]);
end;
end;
end, 100);
end);
end);
end;
procedure TQTXTaskService.HandleGetFileInfo(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lRequest := TQTXFileInfoRequest(Request);
var lUserName := lRequest.UserName.trim();
var lFileName := lRequest.FileName.trim();
// prevent path escape attempts
var lTemp := "../";
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
lTemp := './';
if pos(lTemp, lFileName) > 0 then
begin
SendError(Socket, Request, Format("Unsupported path sequence [%s] detected error", [lTemp]) );
exit;
end;
GetFileInfo(lRequest, lUserName, lFileName,
procedure (TagValue: variant; LocalFile: string; Info: JStats; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.message);
SendError(Socket, Request, Error.Message);
exit;
end;
// Collect the data
var lData := new JFileItem();
lData.diFileName := lFileName;
lData.diFileType := if Info.isFile then JFileItemType.wtFile else JFileItemType.wtFolder;
lData.diFileSize := Info.size;
lData.diFileMode := IntToStr(Info.mode);
lData.diCreated := TDateUtils.FromJsDate( Info.cTime );
lData.diModified := TDateUtils.FromJsDate( Info.mTime );
var lResponse := new TQTXFileInfoResponse(lRequest.Ticket);
lResponse.Routing.TagValue := request.Routing.TagValue;
lResponse.UserName := lUserName;
lResponse.FileName := lFileName;
lResponse.Assign(lData);
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
WriteToLog(e.message);
end;
end);
end;
procedure TQTXTaskService.HandleDestroyDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lMessage := TQTXFileDestroyDeviceRequest(request);
// This will also destroy any files + unregister the device in the
// database table for the service -- do not mess with this!
UnRegisterLocalDevice(nil, lMessage.Username, lMessage.DeviceName,
procedure (TagValue: variant; LocalPath: string; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
var lResponse := TQTXFileDestroyDeviceResponse.Create(request.ticket);
lResponse.UserName := lMessage.UserName;
lResponse.DeviceName := lMessage.DeviceName;
lResponse.Routing.TagValue := Request.Routing.TagValue;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
begin
WriteToLog(e.message);
end;
end;
end);
end;
procedure TQTXTaskService.HandleCreateLocalDevice(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lMessage := TQTXFileCreateLocalDeviceRequest(request);
// Attempt to register.
// NOTE: This will automatically create a matching folder
// under $cwd/userdevices/[calculated_name_of_device]
RegisterLocalDevice(nil, lMessage.Username, lMessage.DeviceName,
procedure (TagValue: variant; LocalPath: string; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
FindDeviceByName(nil, lMessage.Username, lMessage.DeviceName,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
var lResponse := TQTXFileCreateLocalDeviceResponse.Create(request.ticket);
lResponse.UserName := lMessage.UserName;
lResponse.Routing.TagValue := Request.Routing.TagValue;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
if Device nil then
lResponse.assign(Device);
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
begin
WriteToLog(e.message);
end;
end;
end);
end);
end;
procedure TQTXTaskService.HandleGetDeviceByName(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lMessage := TQTXFileGetDeviceByNameRequest(request);
FindDeviceByName(nil, lMessage.Username, lMessage.DeviceName,
procedure (TagValue: variant; Device: JDeviceInfo; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
var lResponse := TQTXFileGetDeviceByNameResponse.Create(request.ticket);
lResponse.UserName := lMessage.UserName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
if Device nil then
lResponse.assign(Device);
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
begin
WriteToLog(e.message);
end;
end;
end);
end;
procedure TQTXTaskService.HandleGetDevices(Socket: TNJWebSocketSocket; Request: TQTXBaseMessage);
begin
var lMessage := TQTXFileGetDeviceListRequest(Request);
GetDevicesForUser(nil, lMessage.Username,
procedure (TagValue: variant; Devices: JDeviceList; Error: Exception)
begin
if Error nil then
begin
WriteToLog(Error.Message);
SendError(Socket, Request, Error.Message);
exit;
end;
var lResponse := TQTXFileGetDeviceListResponse.Create(request.ticket);
lResponse.UserName := lMessage.UserName;
lResponse.Code := CNT_MESSAGE_CODE_OK;
lResponse.Response := CNT_MESSAGE_TEXT_OK;
if Devices nil then
lResponse.assign(Devices);
try
Socket.Send( lResponse.Serialize() );
except
on e: exception do
begin
WriteToLog(e.message);
end;
end;
end);
end;
procedure TQTXTaskService.AfterServerStarted;
begin
inherited;
// Check prefs if zconfig should be applied
if self.FPrefs.ReadBoolean("zconfig", "active", false) then
begin
// ZConfig should only run on the master instance.
// We dont want to register our endpoint for each worker
if NodeJSClusterAPI().isWorker then
exit;
writeln("Setting up Zero-Configuration layer");
FZConfig.port := FPrefs.ReadInteger('zconfig', 'bindport', 2109);
FZConfig.address := GetMachineIP();
FZConfig.Start(nil, procedure (Sender: TObject; TagValue: variant; Error: Exception)
begin
if FPrefs.ReadBoolean("zconfig", "broadcast", true) then
FZConfig.Socket.setBroadcast(true);
// Build up the endpoint (URL) for our websocket server
var lEndpoint := '';
if FPrefs.ReadBoolean('networking', 'secure', false) then
lEndpoint := 'wss://'
else
lEndpoint := 'ws://';
lEndpoint += GetMachineIP();
lEndpoint += ':' + Port.ToString();
// Ping the ZConfig service on interval, until our service is registered
// We keep track of the interval handle so we can stop calling on interval later
FRegHandle := TQTXDispatch.SetInterval( procedure ()
begin
inc(FRegCount);
// Only output once to avoid overkill in the log
if FRegCount = 1 then
WriteToLogF("ZConfig registration begins [%s]", [lEndpoint]);
FZConfig.RegisterService(nil, CNT_ZCONFIG_SERVICE_NAME, SERVICE_ID_TASKMANAGER, lEndpoint,
procedure (TagValue: variant; Error: Exception)
begin
if Error = nil then
begin
WriteToLog("Service registered");
TQTXDispatch.ClearInterval(FRegHandle);
FRegCount := 0;
exit;
end;
end);
end, 1000);
end);
end;
end;
procedure TQTXTaskService.BeforeServerStopped;
begin
inherited;
end;
procedure TQTXTaskService.Dispatch(Socket: TNJWebSocketSocket; Message: TQTXBaseMessage);
begin
var LInfo := MessageDispatch.GetMessageInfoForClass(Message);
if LInfo nil then
begin
try
LInfo.MessageHandler(Socket, Message);
except
on e: exception do
begin
//Log error
WriteToLog(e.message);
end;
end;
end;
end;
end.
Quartex “Cloud Ripper” hardware
For close to a year now I have been busy on a very exciting project, namely my own cloud system. While I have written about this project quite a bit these past months, mostly focusing on the software aspect, not much has been said about that hardware.
Quartex “Cloud Ripper” running neatly on my home-office desk
So let’s have a look at Cloud Ripper, the official hardware setup for Quartex Media Desktop.
Tiny footprint, maximum power
Despite its complexity, the Quartex Media Desktop architecture is surprisingly lightweight. The services that makes up the baseline system (read: essential services) barely consume 40 megabytes of ram per instance (!). And while there is a lot of activity going on between these services -most of that activity is message-dispatching. Sending messages costs practically nothing in cpu and network terms. This will naturally change the moment you run your cloud as a public service, or setup the system in an office environment for a team. The more users, the more signals are shipped between the processes – but with the exception of reading and writing large files, messages are delivered practically instantaneous and hardly use CPU time.
Quartex Media Desktop is based on a clustered micro-service architecture
One of the reasons I compile my code to JavaScript (Quartex Media Desktop is written from the ground up in Object Pascal, which is compiled to JavaScript) has to do with the speed and universality of node.js services. As you might know, Node.js is powered by the Google V8 runtime engine, which means the code is first converted to bytecodes, and further compiled into highly optimized machine-code [courtesy of llvm]. When coded right, such Javascript based services execute just as fast as those implemented in a native language. There simply are no perks to be gained from using a native language for this type of work. There are however plenty of perks from using Node.js as a service-host:
- Node.js delivers the exact same behavior no matter what hardware or operating-system you are booting up from. In our case we use a minimal Linux setup with just enough infrastructure to run our services. But you can use any OS that supports Node.js. I actually have it installed on my Android based Smart-TV (!)
- We can literally copy our services between different machines and operating systems without recompiling a line of code. So we don’t need to maintain several versions of the same software for different systems.
- We can generate scripts “on the fly”, physically ship the code over the network, and execute it on any of the machines in our cluster. While possible to do with native code, it’s not very practical and would raise some major security concerns.
- Node.js supports WebAssembly, you can use the Elements Compiler from RemObjects to write service modules that executes blazingly fast yet remain platform and chipset independent.
The Cloud-Ripper cube
The principal design goal when I started the project, was that it should be a distributed system. This means that instead of having one large-service that does everything (read: a typical “native” monolithic design), we instead operate with a microservice cluster design. Services that run on separate SBC’s (single board computers). The idea here is to spread the payload over multiple mico-computers that combined becomes more than the sum of their parts.
Cloud Ripper – Based on the Pico 5H case and fitted with 5 x ODroid XU4 SBC’s
So instead of buying a single, dedicated x86 PC to host Quartex Media Desktop, you can instead buy cheap, off-the-shelves, easily available single-board computers and daisy chain them together. So instead of spending $800 (just to pin a number) on x86 hardware, you can pick up $400 worth of cheap ARM boards and get better network throughput and identical processing power (*). In fact, since Node.js is universal you can mix and match between x86, ARM, Mips and PPC as you see fit. Got an older PPC Mac-Mini collecting dust? Install Linux on it and get a few extra years out of these old gems.
(*) A single XU4 is hopelessly underpowered compared to an Intel i5 or i7 based PC. But in a cluster design there are more factors than just raw computational power. Each board has 8 CPU cores, bringing the total number of cores to 40. You also get 5 ARM Mali-T628 MP6 GPUs running at 533MHz. Only one of these will be used to render the HTML5 display, leaving 4 GPUs available for video processing, machine learning or compute tasks. Obviously these GPUs won’t hold a candle to even a mid-range graphics card, but the fact that we can use these chips for audio, video and computation tasks makes the system incredibly versatile.
Another design goal was to implement a UDP based Zero-Configuration mechanism. This means that the services will find and register with the core (read: master service) automatically, providing the machines are all connected to the same router or switch.
Put together your own supercomputer for less than $500
The first “official” hardware setup is a cluster based on 5 cheap ARM boards; namely the ODroid XU4. The entire setup fits inside a Pico Cube, which is a special case designed to house this particular model of single board computers. Pico offers several different designs, ranging from 3 boards to a 20 board super-cluster. You are not limited ODroid XU4 boards if you prefer something else. I picked the XU4 boards because they represent the lowest possible specs you can run the Quartex Media Desktop on. While the services themselves require very little, the master board (the board that runs the QTXCore.js service) is also in charge of rendering the HTML5 display. And having tested a plethora of boards, the ODroid XU4 was the only model that could render the desktop properly (at that low a price range).
Note: If you are thinking about using a Raspberry PI 3B (or older) as the master SBC, you can pretty much forget it. The media desktop is a piece of very complex HTML5, and anything below an ODroid XU4 will only give you a terrible experience (!). You can use smaller boards as slaves, meaning that they can host one of the services, but the master should preferably be an ODroid XU4 or better. The ODroid N2 [with 4Gb Ram] is a much better candidate than a Raspberry PI v4. A Jetson Nano is an even better option due to its extremely powerful GPU.
Booting into the desktop
One of the things that confuse people when they read about the desktop project, is how it’s possible to boot into the desktop itself and use Quartex Media Desktop as a ChromeOS alternative?
How can a “cloud platform” be used as a desktop alternative? Don’t you need access to the internet at all times? If it’s a server based system, how then can we boot into it? Don’t we need a second PC with a browser to show the desktop?
Accessing the desktop like a “web-page” from a normal Linux setup
To make a long story short: the “master” in our cluster architecture (read: the single-board computer defined as the boss) is setup to boot into a Chrome browser display under “kiosk mode”. When you start Chrome in kiosk mode, this removes all traces of the ordinary browser experience. There will be no toolbars, no URL field, no keyboard shortcuts, no right-click popup menus etc. It simply starts in full-screen and whatever HTML5 you load, has complete control over the display.
What I have done, is to to setup a minimal Linux boot sequence. It contains just enough Linux to run Chrome. So it has all the drivers etc. for the device, but instead of starting the ordinary Linux Desktop (X or Wayland) -we instead start Chrome in kiosk mode.
Booting into the same desktop through Chrome in Kiosk Mode. In this mode, no Linux desktop is required. The Linux boot sequence is altered to jump straight into Chrome
Chrome is started to load from 127.0.0.1 (this is a special address that always means “this machine”), which is where our QTXCore.js service resides that has it’s own HTTP/S and Websocket servers. The client (HTML5 part) is loaded in under a second from the core — and the experience is more or less identical to starting your ChromeBook or NAS box. Most modern NAS (network active storage) devices are much more than a file-server today. NAS boxes like those from Asustor Inc have HDMI out, ships with a remote control, and are designed to act as a media center. So you connect the NAS directly to your TV, and can watch movies and listen to music without any manual conversion etc.
In short, you can setup Quartex Media Desktop to do the exact same thing as ChromeOS does, booting straight into the web based desktop environment. The same desktop environment that is available over the network. So you are not limited to visiting your Cloud-Ripper machine via a browser from another computer; nor are you limited to just using a dedicated machine. You can setup the system as you see fit.
Why should I assemble a Cloud-Ripper?
Getting a Cloud-Ripper is not forced on anyone. You can put together whatever spare hardware you have (or just run it locally under Windows). Since the services are extremely lightweight, any x86 PC will do. If you invest in a ODroid N2 board ($80 range) then you can install all the services on that if you like. So if you have no interest in clustering or building your own supercomputer, then any PC, Laptop or IOT single-board computer(s) will do. Provided it yields more or equal power as the XU4 (!)
What you will experience with a dedicated cluster, regardless of putting the boards in a nice cube, is that you get excellent performance for very little money. It is quite amazing what $200 can buy you in 2019. And when you daisy chain 5 ODroid XU4 boards together on a switch, those 5 cheap boards will deliver the same serving power as an x86 setup costing twice as much.
The NVidia Jetson Nano SBC, one of the fastest boards available at under $100
Pico is offering 3 different packages. The most expensive option is the pre-assembled cube. This is for some reason priced at $750 which is completely absurd. If you can operate a screwdriver, then you can assemble the cube yourself in less than an hour. So the starter-kit case which costs $259 is more than enough.
Next, you can buy the XU4 boards directly from Hardkernel for $40 a piece, which will set you back $200. If you order the Pico 5H case as a kit, that brings the sub-total up to $459. But that price-tag includes everything you need except sd-cards. So the kit contains power-supply, the electrical wiring, a fast gigabit ethernet switch [built-into the cube], active cooling, network cables and power cables. You don’t need more than 8Gb sd-cards, which costs practically nothing these days.
Note: The Quartex Media Desktop “file-service” should have a dedicated disk. I bought a 256Gb SSD disk with a USB 3.0 interface, but you can just use a vanilla USB stick to store user-account data + user files.
As a bonus, such a setup is easy to recycle should you want to do something else later. Perhaps you want to learn more about Kubernetes? What about a docker-swarm? A freepascal build-server perhaps? Why not install FreeNas, Plex, and a good backup solution? You can set this up as you can afford. If 5 x ODroid XU4 is too much, then get 3 of them instead + the Pico 3H case.
So should Quartex Media Desktop not be for you, or you want to do something else entirely — then having 5 ODroid XU4 boards around the house is not a bad thing.
Oh and if you want some serious firepower, then order the Pico 5H kit for the NVidia Jetson Nano boards. Graphically those boards are beyond any other SoC on the market (in it’s price range). But as a consequence the Jetson Nano starts at $99. So for a full kit you will end up with $500 for the boards alone. But man those are the proverbial Ferrari of IOT.
Quartex Media Desktop, new compiler and general progress
It’s been a few weeks since my last update on the project. The reason I dont blog that often about Quartex Media Desktop (QTXMD), is because the official user-group has grown to 2000+ members. So it’s easier for me to post developer updates directly to the audience rather than writing articles about it.
Quartex Media Desktop ~ a complete environment that runs on every device
If you haven’t bothered digging into the project, let me try to sum it up for you quickly.
Quick recap on Quartex Media Desktop
To understand what makes this project special, first consider the relationship between Microsoft Windows and a desktop program. The operating system, be it Windows, Linux or OSX – provides an infrastructure that makes complex applications possible. The operating-system offers functions and services that programs can rely on.
The most obvious being:
- A filesystem and the ability to save and load data
- A windowing toolkit so programs can be displayed and have a UI
- A message system so programs can communicate with the OS
- A service stack that takes care of background tasks
- Authorization and identity management (security)
I have just described what the Quartex Media Desktop is all about. The goal is simple:
to provide for JavaScript what Windows and OS X provides for ordinary programs.
Just stop and think about this. Every “web application” you have ever seen, have all lacked these fundamental features. Sure you have libraries that gives you a windowing environment for Javascript, like Embarcadero Sencha; but im talking about something a bit more elaborate. Creating windows and buttons is easy, but what about ownership? A runtime environment has to keep track of the resources a program allocates, and make sure that security applies at every step.
Target audience and purpose
Take a second and think about how many services you use that have a web interface. In your house you probably have a router, and all routers can be administered via the browser. Sadly, most routers operate with a crude design and that leaves much to be desired.
Router interfaces for web are typically very limited and plain looking. Imagine what NetGear could do with Quartex Media Desktop instead
If you like to watch movies you probably have a Plex or Kodi system running somewhere in your house; perhaps you access that directly via your TV – or via a modern media system like Playstation 4 or XBox one. Both Plex and Kodi have web-based interfaces.
Netflix is now omnipresent and have practically become an institution in it’s own right. Netflix is often installed as an app – but the app is just a thin wrapper around a web-interface. That way they dont have to code apps for every possible device and OS out there.
If you commute via train in Scandinavia, chances are you buy tickets on a kiosk booth. Most of these booths run embedded software and the interface is again web based. That way they can update the whole interface without manually installing new software on each device.
Plex is a much loved system. It is based on a mix of web and native technologies
These are just examples of web based interfaces you might know and use; devices that leverage web technology. As a developer, wouldn’t it be cool if there was a system that could be forked, adapted and provide advanced functionality out of the box?
Just imagine a cheap Jensen router with a Quartex Media Desktop interface! It could provide a proper UI interface with applications that run in a windowing environment. They could disable ordinary desktop functionality and run their single application in kiosk mode. Taking full advantage of the underlying functionality without loss of security.
And the same is true for you. If you have a great idea for a web based application, you can fork the system, adjust it to suit your needs – and deploy a cutting edge cloud system in days rather than months!
New compiler?
Up until recently I used Smart Mobile Studio. But since I have left that company, the matter became somewhat pressing. I mean, QTXMD is an open-source system and cant really rely on third-party intellectual property. Eventually I fired up Delphi, forked the latest DWScript, and used that to roll a new command-line compiler.
Web technology has reached a level of performance that rivals native applications. You can pretty much retire Photoshop in favour of web based applications these days
But with a new compiler I also need a new RTL. Thankfully I have been coding away on the new RTL for over a year, but there is still a lot of work to do. I essentially have to implement the same functionality from scratch.
There will be more info on the new compiler / codegen when its production ready.
Progress
If I was to list all the work I have done since my last post, this article would be a small book. But to sum up the good stuff:
- Authentication has been moved into it’s own service
- The core (the main server) now delegates login messages to said service
- We no longer rely on the Smart Pascal filesystem drivers, but use the raw node.js functions instead (faster)
- The desktop now use the Smart Theme engine. This means that we can style the desktop to whatever we like. The OS4 theme that was hardcoded will be moved into its own proper theme-file. This means the user can select between OS4, iOS, Android and Ubuntu styling. Creating your own theme-files is also possible. The Smart theme-engine will be replaced by a more elaborate system in QTX later
- Ragnarok (the message api) messages now supports routing. If a routing structure is provided, the core will relay the message to the process in question (providing security allows said routing for the user)
- The desktop now checks for .info files when listing a directory. If a file is accompanied by an .info file, the icon is extracted and shown for that file
- Most of the service layer now relies on the QTX RTL files. We still have some dependencies on the Smart Pascal RTL, but we are making good progress on QTX. Eventually the whole system will have no dependencies outside QTX – and can thus be compiled without any financial obligations.
- QTX has it’s own node.js classes, including server and client base-classes
- Http(s) client and server classes are added to QTX
- Websocket and WebSocket-Secure are added to QTX
- TQTXHybridServer unifies http and websocket. Meaning that this server type can handle both orinary http requests – but also websocket connections on the same network socket. This is highly efficient for websocket based services
- UDP classes for node.js are implemented, both client and server
- Zero-Config classes are now added. This is used by the core for service discovery. Meaning that the child services hosted on another machine will automatically locate the core without knowing the IP. This is very important for machine clustering (optional, you can define a clear IP in the core preferences file)
- Fixed a bug where the scrollbars would corrupt widget states
- Added API functions for setting the scrollbars from hosted applications (so applications can tell the desktop that it needs scrollbar, and set the values)
- .. and much, much more
I will keep you all posted about the progress — the core (the fundamental system) is set for release in december – so time is of the essence! Im allocating more or less all my free time to this, and it will be ready to rock around xmas.
When the core is out, I can focus solely on the applications. Everything from Notepad to Calculator needs to be there, and more importantly — the developer tools. The CloudForge IDE for developers is set for 2020. With that in place you can write applications for iOS, Android, Windows, OS X and Linux directly from Quartex Media Desktop. Nothing to install, you just need a modern browser and a QTX account.
The system is brilliant for small teams and companies. They can setup their own instance, communicate directly via the server (text chat and video chat is scheduled) and work on their products in concert.
New job, new office, new adventures
It’s been roughly 4 weeks since I posted a status report on Amibian.js. I normally keep people up-to-date on facebook (the “Amiga Disrupt” and also “Delphi Developer” groups). It’s been a very hectic month so I fully understand that people are asking. So let’s look at where the project is at and where we are on the time-line.
For those that might not know, I decided to leave Embarcadero a couple of months ago. I will be working out may before I move on. I wanted to write about that myself in a clean fashion, but sadly the news broke on Facebook prematurely.
Long story short, I have been very fortunate to work at Embarcadero. I am not leaving because there is anything wrong or something like that. I was hired as SC for the EMEA regions, which basically made me the support and presenter for most of europe, parts of asia and the middle east. It’s been a great adventure, but ultimately I had to admit that my passion is coding and community work. Sales is a very important part of any company, but it’s not really my cup of tea; my passion has always been research and development.
So, come first of June and I start in a new position at RemObjects. A company that has deep roots with Delphi and C++ builder users – and a company that continues to produce a wealth of high-quality, high-performance frameworks for Delphi and C++ builder. RemObjects also has a strong focus on modern languages, and have a strong portfolio of new and exciting compilers and languages to offer. The Oxygene compiler should be no stranger to Delphi developers, a powerful object-pascal dialect that can target a variety of platforms and chipsets.
Since compiler technology and run-time systems has been my main focus for well over a decade now, I feel RemObjects is a better match.
Quartex Components
Quartex Components has been an officially registered Norwegian company for a while now, so perhaps not news. What is news is that it’s now directly connected with the development of the Quartex Media Desktop (codename “Amibian.js”). While Amibian.js is an open source endeavour, there will be both free and commercial products running on top of that platform. I have written at length about Cloud Forge in the past, so I wont re-hash that again. But 2020 will see a paradigm shift in how teams and companies approach software development.
Company logo professionally milled and on its way to my new office
I will also, once there is more time, continue to sell and support software license components.
Quartex Media Desktop
The “Amibian.js” project is moving along nicely. The deadline is Q4 2019, but im hoping to wrap up the core functionality before that. So we are on track and kicking ass 🙂
More and more elaborate functionality is being implemented for the desktop
Here is an overview of work done this month:
- TSystemService application type has been created (node.js)
- TApplication now holds IPC functions (inter process communication)
- Running child processes + sending messages is now simplicity itself
- Database drivers are 90% done. Delete() and DeleteTable() functionality needs to be implemented in a uniform way
- Authentication is now a separate service
- Service database layer is finished (using SQLite3 driver by default)
- Authentication protocol has been designed
- Server protocol and JSON message envelopes are done
- Presently working on the client interface
- LDEF bytecode assembler has been improved
- Faster symbolic lookup
- Smarter register recognition
- Early support for stack-frames
- Fixed bug in parser (comma-list parse)
- QTX framework has seen a lot of work
- Large parts of the RTL sub-strata has been implemented
- UTF16 codec implemented
- QTX versions of common controls:
- TQTXButton
- TQTXLabel
- TQTXToolbar
- TQTXToolButton
- TQTXToolSeparator
- TQTXToolElement
- TQTXPanel
- TQTXCheckBox
- .. and much, much more
- Desktop changes
- Link Maker functionality has been added
- Handshake process between desktop and child app now runs on a separate timer, ensuring better conformity and a more robust initialization
- The Quartex Editor control has been optimized
- All redraw calls are now synchronized
- Canvas is created on demand, avoids flicker during initial redraw
- Support for DEL key + behavior
- Gutter is now rendered to an offscreen bitmap and blitted into the control’s canvas. The gutter is only fully rendered when cursor forces the view to change
I will continue to keep everyone up to date about the project. As you can understand, its a bit hectic right now so please be patient – it is turning into an EPIC environment!
Quartex: Mali GPU glitches
EDIT: I did further testing after this article was written, and believe the source of this to be about heat. Even with extra fans, running games like Tyrian (asm.js) that are extremely demanding, plus resizing a graphics intensive windows constantly, the temperature reached 71 degrees C very quickly. And this was with two cabinet fans helping the built-in fan to cool the device. It is thus not unthinkable that when running solo (no extra fans) that the kernel shut the device down to not cook the chipset. Which also explains why the device wont boot properly afterwards (the device is still hot).
Glitches
Something really strange is happening on Chrome and Firefox for ARM. JavaScript is not supposed to be able to take down a system, and in this case it’s neither an attempt as such either — yet for some reason I have managed to take down the ODroid XU4 with both Chrome and Firefox lately.
ODroid XU4
I guess I should lead with that I’m not able to replicate this on x86. One of the things I really love about the ODroid XU4 is that it’s affordable, powerful and probably the only SBC I have used that runs stable on the mali GPU. As you probably know I tested at least 10 different SBC’s back in 2018, and whenever there was a mali GPU involved, the product was either haunted by instabilities or lacked drivers all together.
Since the codebase for Chrome (and I presume Firefox) is ultimately the same between platforms, it leaves a question-mark about the ODroid. It is by far the most stable SBC I have tested so far (except for the PI, which is sadly underpowered for this task), but stable doesn’t mean flawless. And to be honest, Amibian.js is pushing web tech to the very limits.
Not Mali again
The reason I suspect the mali to be the culprit behind all this, is because the “bug” if we can call it that, happens exclusively during resize. So if there is a lot going on inside a desktop-window, you can sometimes provoke the ODroid to cold-crash and reboot. You actually have to power the board down and switch it back on for it to boot properly.
Cloudripper ~ 5x ODroid XU4 [40 cores] in a PICO 5h cube
The challenge is that there is no way to debug or catch this error, because when it occurs the whole system literally goes down. There is no exception thrown, nor is the browser process terminated (not even a log entry, so it’s a clean-cut) — the system reboots on the spot. Since it fails on reboot when opening X (setting a screen-mode) I again point the finger at the GPU. Somehow a flag or lock survives the cold-reboot and that’s why you have to manually switch it off and on again.
This is the exact problem that made the NanoPI Fire useless. It only shipped with Android embedded drivers. The X drivers could hardly open a display without crashing. Such a waste of a good cpu.
x86 as head
ODroid is perfect for a low-cost Amibian.js experience, but I was unsure if it would handle the payload. Interestingly it handles it just fine and even with a high-speed action game running + background tasks we are not using 50% of the CPU even.
Ram is holding up too, with memory consumption while running Tyrian + having a few graphics viewers open, is at a reasonable 700 mb (of 2 gigabyte in total).
Tyrian jogs along at 45 fps ~ that is not bad for a $45 SBC
Right now this strange error is rare, but if it continues or grows into a problem (chrome is hardly useable at all, only firefox) then I have no option than to replace the master sbc in the cluster with something else. The x86 UP board is more than capable, but it would be a shame to break the price range because of that (excuse my language) crap mali GPU. I honestly don’t understand why board makers insist on using a mali. Every board that has a mali is haunted by problems and get poor reviews.
It will be exciting to check out the dragonboard, although I fear 1Gb memory will not be enough for smooth operation. Not without a sata interface and a good swap-file.
Android and Delphi
One alternative is to switch to Android and use Delphi to code a custom Chromium Embedded webview. I am hoping to avoid the overhead of Android, but Delphi would definitively be a bonus with Android embedded (“Android of things”).
We will see.
Smart Pascal + WebOS = true
If you own a television (and who doesn’t) chances are you own an LG model. The LG brand has been on the market since before recorded history (or so it feels) and have been a major player for decades.
What is little known though, is that LG also owns and finance a unique operating system. This is not uncommon these days, most NAS and cloud vendors have their own system (there are roughly 20 cloud based systems on the market that I know of). The only way to make a Smart TV (sigh) is to add a computer to it. And if you buy a television today chances are it contains a small embedded board running some custom-made operating system designed to do just that.
Every vendor has their own system, and those that don’t usually end up forking Android and adapt that to their needs.
Luna WebOS
LG’s system is called WebOS. This may sound like yet another html eye candy front end, but hear me out because this OS is not what it seems.
Except for some font issues (see disk label) Smart loaded fine under WebOS
Remember Palm OS? If you are between 35 and 45 you should remember that before Apple utterly demolished the mobile-phone market with their iPhone, one of the most popular brands was Palm. Their core product being “Palm Pilot”, a kind of digital filo-fax (personal organizer) and mobile phone rolled into one.
WebOS is based on what used to be called Palm-OS, but it has been completely revamped, given a sexy new user interface (looks a lot like android to be honest!) and brought into the present age. And best of all: its 100% free! It runs on a plethora of systems, from x86 to Raspberry PI to Mips. It is used in televisions, terminals and set-top-boxes around the world – and is quite popular amongst engineers.
Check it out their new portal here; there is also plenty of links to pre-built images if you look around there: https://pivotce.com/
JavaScript application stack
One of the coolest features in my view, is that their applications are primarily made by JavaScript. The OS itself is native of course, but they have discovered that JavaScript and HTML5 is an excellent way to build applications. Applications that is easy to control, sandboxed and safe yet incredibly powerful and capable.
Smart Desktop booting Quake 3 in Luna OS
Well, today I sent them an Email requesting their SDK. I know they use Enyo.js as their primary and suggested framework – but that is no problem for Smart Mobile Studio. A better question is: can Luna handle our codebase?
When I loaded up Quake 3 in pure Asm.JS the TV crashed with a spectacular access violation. So they are probably not used to the level of hardcore coding Smart developers represent. But yeah, Quake III is about as advanced as it gets – and it pushes any browser to the outer limits of what is possible.
Once I get the SDK, docs and a few examples – I will begin adding support for it as quickly as possible. Which means you get a new project type especially for the platform + API units (typically stored under $RTL\Apis folder).
Why is this cool?
Because with support for Luna / WebOS, you as a Delphi or Smart developer can offer your services to companies that use WebOS or Luna (the open source version) in their devices. The list of companies is substantial – and they are well established companies. And as you have no doubt noticed, hardware engineers doesn’t always make the best software engineers. So this opens up plenty of opportunities for a good object pascal / Smart developer.
Luna has an Android like quality over it – except its more smooth to use
Remember – other developers will use vanilla JavaScript. You have the onslaught of the VJL and the might of our libraries at your disposal. You can produce in days what others use weeks on achieving.
These are exciting days! I’ll keep you posted on our progress!
Smart Pascal, the next generation
I want to take the time to talk a bit about the future, because like all production companies we are all working towards lesser and greater goals. If you don’t have a goal then you are in trouble; Thankfully our goals have been very clear from the beginning. Although I must admit that our way there has been.. “colorful” at times.
When we started back in 2010 we didn’t really know what would become of our plans. We only knew that this was important; there was a sense of urgency and “we have to build this” in the air; I think everyone involved felt that this was the case, without any rational explanation as to why. Like all products of passion it can consume you in a way – and you work day and night on turning an idea into something real. From the intangible to the tangible.
It seems like yesterday, but it was 5 years ago!
By the end of 2011 / early 2012, Eric and myself had pretty much proven that this could be done. At the time there were more than enough nay-sayers and I think both of us got flamed quite often for daring to think different. People would scoff at me and say I was insane to even contemplate that object pascal could ever be implemented for something as insignificant and mediocre as JavaScript. This was my first meeting with a sub-culture of the Delphi and C++ community, a constellation I have gone head-to-head with on many occasions. But they have never managed to shake my resolve as much as inch.
When we released version 1.0 in 2012 some ideas about what could be possible started to form. Jørn defined plans for a system we later dubbed “Smart net”. In essence it would be something you logged onto from the IDE – allowing you to store your projects in the cloud, compile in the cloud (connected with Adobe build services) and essentially move parts of your eco-system to the cloud. Keep in mind this was when people still associated cloud with “storage”; they had not yet seen things like Uber or Netflix or played Quake 3 at 160 frames per second, courtesy of asm.js in their browser.
The second part would be a website where you could do the same, including a live editor, access to the compiler and also the ability to buy and sell components, solutions and products. But for that we needed a desktop environment (which is where the Quartex Media Desktop came in later).
The first version of the Media Desktop, small but powerful. Here running in touch-screen mode with classical mobile device layout (full screen forms).
Well, we have hit many bumps along the road since then. But I must be honest and say, some of our detours have also been the most valuable. Had it not been for the often absurd (to the person looking in) research and demo escapades I did, the RTL wouldn’t be half as powerful as it is today. It would deliver the basics, perhaps piggyback on Ext.js or some lame, run of the mill framework – and that would be that. Boring, flat and limited.
What we really wanted to deliver was a platform. Not just a website, but a rich environment for creating, delivering and enjoying web and cloud based applications. And without much fanfare – that is ultimately what the Smart Desktop and it’s sexy node.js back-end is all about is all about.
We have many project types in the pipeline, but the Smart Desktop type is by far the most interesting and powerful. And its 100% under your control. You can create both the desktop itself as a project – and also applications that should run on that desktop as separate projects.
This is perfectly suited for NAS design (network active storage devices can usually be accessed through a web portal on the device), embedded boards, intranets and even intranets for that matter.
You get to enjoy all the perks of a multi-user desktop, one capable of both remote desktop access, telnet access, sharing files and media, playing music and video (we even compiled the mp4 codec from C to JavaScript so you can play mp4 movies without the need for a server backend).
The Smart Desktop
The Smart Desktop project is not just for fun and games. We have big plans for it. And once its solid and complete (we are closing in on 46% done), my next side project will not be more emulators or demos – it will be to move our compiler(s) to Amazon, and write the IDE itself from scratch in Smart Pascal.
The Smart Desktop – A full desktop in the true sense of the word
And yeah, we have plans for EmScripten as well – which takes C/C++ and compiles it into asm.js. It will take a herculean effort to merge our RTL with their sandboxed infrastructure – but the benefits are too great to ignore.
As a bonus you get to run native 68k applications (read: Amiga applications) via emulation. While I realize this will be mostly interesting for people that grew up with that machine – it is still a testament to the power of Smart and how much you can do if you really put your mind to it.
Naturally, the native IDE wont vanish. We have a few new directions we are investigating here – but native will absolutely not go anywhere. But cloud, the desktop system we are creating, is starting to become what we set out to make five years ago (has it been half a decade already? Tempus fugit!). As you all know it was initially designed as an example of how you could write full-screen applications for Raspberry PI and similar embedded devices. But now its a full platform in its own right – with a Linux core and node.js heart, there really is very little you cannot do here.
The Smart Pascal compiler is one of our tools that is ready for cloud-i-fication
Being able to login to the Smart company servers, fire up the IDE and just code – no matter if you are: be it Spain, Italy, Egypt, China or good old USA — is a pretty awesome thing!
Clicking compile and the server does the grunt work and you can test your apps live in a virtual window; switch between device layouts and targets — then hit “publish” and it goes to Cordova (or Delphi) and voila – you get a message back when binaries for 9 mobile devices is ready for deployment. One click to publish your applications on Appstore, Google play and Microsoft marketplace.
Object pascal works
People may have brushed off object pascal (and from experience those people have a very narrow view of what object pascal is all about), but when they see what Smart delivers, which in itself is written in Delphi, powered by Delphi and should be in every Delphi developer’s toolbox — i think it should draw attention to both Delphi as a product, object pascal as a language – and smart as a solution.
With Smart it doesn’t matter what computer you use. You can sit at home with the new A1222 PPC Amiga, or a kick-ass Intel i7 beast that chew virtual machines for breakfast. If your computer can handle a modern website, then you can learn object pascal and work directly in the cloud.
The Smart Desktop running on cheap embedded hardware. The results are fantastic and the financial savings of using Smart Pascal on the kiosk client is $400 per unit in this case
Heck you can work off a $60 ODroid XU4, it has more than enough horsepower to drive the latest chrome or Firefox engines. All the compilation takes place on the server anyways. And if you want a Delphi vessel rather than phonegap (so that it’s a Delphi application that opens up a web-view in full-screen and expose features to your smart code) then you will be happy to know that this is being investigated.
More targets
There are a lot of systems out there in the world, some of which did not exist just a couple of years ago. FriendOS is a cloud based operating system we really want to support, so we are eager to get cracking on their SDK when that comes out. Being able to target FriendOS from Smart is valuable, because some of the stuff you can do in SMS with just a bit of code – would take weeks to hand write in JavaScript. So you get a productive edge unlike anything else – which is good to have when a new market opens.
As far as Delphi is concerned there are smaller systems that Embarcadero may not be interested in, for example the many embedded systems that have come out lately. If Embarcadero tried to target them all – it would be a never-ending cat and mouse game. It seems like almost every month there is a new board on the market. So I fully understand why Embarcadero sticks to the most established vendors.
Smart technology allows you to cover all your bases regardless of device
But for you, the programmer, these smaller boards can repsent thousands of dollars worth of saving. Perhaps you are building a kiosk system and need to have a good-looking user interface that is not carved in stone, touch capabilities, low-latency full-duplex communication with a server; not much you can do about that if Delphi doesnt target it. And Delphi is a work horse so it demands a lot more cpu than a low-budget ARM SoC can deliver. But web-tech can thrive in these low-end environments. So again we see that Smart can compliment and be a valuable addition to Delphi. It helps you as a Delphi developer to act on opportunities that would otherwise pass you by.
So in the above scenario you can double down. You can use Smart for the user-interface on a low power, low-cost SoC (system on a chip) kiosk — and Delphi on the server.
It all depends on what you are interfacing with. If you have a full Delphi backend (which I presume you have) then writing the interface server in Delphi obviously makes more sense.
If you don’t have any back-end then, depending on your needs or future plans, it could be wise to investigate if node.js is right for you. If it’s not – go with what you know. You can make use of Smart’s capabilities either way to deliver cost-effective, good-looking device front-ends of mobile apps. Its valuable tool in your Delphi toolbox.
Better infrastructure and rooting
So far our support for various systems has been in the form of APIs or “wrapper units”. This is good if you are a low-level coder like myself, but if you are coming directly from Delphi without any background in web technology – you wouldn’t even know where to start.
So starting with the next IDE update each platform we support will not just have low-level wrapper units, but project types and units written and adapted by human beings. This means extra work for us – but that is the way it has to be.
As of writing the following projects can be created:
- HTML5 mobile applications
- HTML5 mobile console applications
- Node.js console applications
- node.js server applications
- node.js service applications (requires PM2)
- Web worker project (deprecated, web-workers can now be created anywhere)
We also have support for the following operating systems:
- Chrome OS
- Mozilla OS
- Samsung Tizen OS
The following API’s have shipped with Smart since version 1.2:
- Khronos browser extensions
- Firefox spesific API
- NodeWebkit
- Phonegap
- Phonegap provides native access to roughly 9 operating systems. It is however cumbersome to work with and beta-test if you are unfamiliar with the “tools of the trade” in the JavaScript world.
- Whatwg
- WAC Apis
Future goals
The first thing we need to do is to update and re-generate ALL header files (or pascal units that interface with the JavaScript libraries) and make what we already have polished, available, documented and ready for enterprise level use.
Why pay $400 to power your kiosk when $99 and Smart can do a better job?
Secondly, project types must be established where they make sense. Not all frameworks are suitable for full project isolation, but act more like utility libraries (like jQuery or similar training-wheels). And as much as possible of the RTL made platform independent and organized following our namespace scheme.
But there are also other operating systems we want to support:
- Norwegian made Friend OS, which is a business oriented cloud desktop
- Node.js OS is very exciting
- LG WebOS, and their Enyo application framework
- Asustor DLM web operating system is also a highly attractive system to support
- OpenNAS has a very powerful JavaScript application framework
- Segate Nas OS 4 likewise use JavaScript for visual, universal applications
- Microsoft Universal Platform allows you to create truly portable, native speed JavaScript applications
- QNap QTS web operating system [now at version 4.2]
All of these are separate from our own NAS and embedded device system: Smart Desktop, which uses node.js as a backend and will run on anything as long as node and a modern browser is present.
Final words
I hope you guys have enjoyed my little trip down memory lane, and also the plans we have for the future. Personally I am super excited about moving the IDE to the cloud and making Smart available 24/7 globally – so that everyone can use it to design, implement and build software for the future right now.
Smart Pascal Builder (or whatever nickname we give it) is probably the first of its kind in the world. There are a ton of “write code on the web” pages out there, but so far there is not a single hard-core development studio like I have in mind here.
So hold on, because the future is just around the corner 😉
Smart desktop: Amibian.js past, future and present
Had someone told me 20 years ago that I would enter my 40’s enjoying JavaScript, my younger self would probably have beaten that someone over the head with a book on hardcore demo coding or something. So yeah, things have changed so much and we are right now in the middle of a paradigm shift that is taking us all to the next level – regardless if we agree or not.
Ask a person what he thinks about “cloud” and you rarely get an answer that resembles what cloud really is. Most will say “its a fancy way of dealing with storage”. Others will say its all about web-services – and others might say it’s about JavaScript.
Old coders never die, we just get better
They are all right and wrong at the same time. Cloud is first of all the total abstraction of all parts that makes up a networked computer system. Storage, processing capacity, memory, operating system, services, programming language and even instruction set.
Where a programmer today works with libraries and classes to create programs that run on a desktop — dragging visual controls like edit-boxes and buttons into place using a form or window designer; a cloud developer builds whole infrastructures. He drags and drops whole servers into place, connects external and internal services.
Storage? Ok I need Dropbox, amazon, Google drive, Microsoft one disk, local disk – and he just drags that onto a module. Done. All of these services now have a common API that allows them to talk with each other. They become like dll files or classes, all built using the same mold – but with wildly different internals. It doesn’t matter as long as they deliver the functionality according to standard.
Processing power? Setup an Azure or Amazon account and if you got the cash, you can compute enough to pre-cacalculate the human brain if you like. It has all been turned into services — that you can link together like lego pieces.
Forget everything you think you know about web; that is but the visual rendering engine over the proverbial death-star of technology hidden beneath. People have only seen the tip of the ice berg.
Operating systems have been reduced to a preference. There is no longer any reason to pick Windows over Linux on the server. Microsoft knew years ago that this day would come. Back in the late 90s I remember reading an interview with Steve Balmer doing one of his black-ops meetings with Norwegian tech holders in Oslo; and he outlined software as a service when people were on 14.4 modems. He also mentioned that “we need a language that is universal” to make this a reality. You can guess why .net was stolen from Borland, not to mention their failed hostile takover of Java (or J#) which Anders Hejlsberg was hired to spear-head.
Amibian.js
Amibian.js is my, Gunnar and Thomas‘s effort to ensure that the Amiga is made portable and can be enjoyed in this new paradigm. It is not made to compete with anyone (as was suggested earlier), but rather to make sure Amiga gets some life into her again – and that people of this generation and the kids growing up now can get to enjoy the same exciting environment that we had.
From Scandinavia with love
The world is going JavaScript. Hardware now speaks JavaScript (!), your TV now speaks JavaScript – heck your digital watch probably runs JavaScript. And just to add insult to injury – asm.js now compiles JS code side-by-side with C/C++ in terms of speed. I think the browser has seen more man years of development time than any other piece of software out there – with the exception of GCC / Gnu Linux perhaps.
Amibian is also an example of a what you can do with Smart Pascal, which is a programming environment that compiles object pascal to JavaScript. One we (The Smart Company AS) made especially for this new paradigm. I knew this was coming years ago – and have been waiting for it. But that’s another story all together.
Future
Well, naturally the desktop system is written from scratch so it needs to be completed. We are at roughly 40% right now. But there is also work to be done on UAE.js (a mild fork of sae, scriptable Amiga emulator) in terms of speed and IO. We want the emulated Amiga side to interact with the modern web desktop – to be able to load and save files to whatever backend you are using.
For those about to rock; We salute you!
Well, it’s not that easy. UAE is like an organism, and introducing new organs into an existing creature is not easily done. UAE.js (or SAE) has omitted a huge chunk of the original code – which also includes the modified boot-code that adds support for external or “virtual” UAE drives (thanks to Frode Solheim of Fs-UAE for explaining the details of the parts here).
But, there are hacker ways. The dark side is a pathway to many abilities, some deemed unnatural. So if all else fails, i will kidnap the hardfile API and delegate the IO signals to the virtual filesystem on the server — in short, UAE.JS will think it’s booting from a hardfile in memory – when in reality it will get its data from node.js.
There are some challenges here. UAE (the original) is not async but ordinary, linear C code. JavaScript is async and may not return the data on exit of the method. So i suspect I will have to cache quite a lot. Perhaps as much as 1 megabyte backwards and forwards from the file-position. But getting data in there we will (and out), come hell or high water.
We can also drop a lot of sub code, like parts of the gayle interface. I found out this is the chip that deals with the IDE interface — so it has to be there, but it can also host memory expansions – but who the hell cares in 2017 in JavaScript about that. More than enough fun via standard chip/fast/rtg memory – so the odd bits can be removed.
So we got our work cut out for us. But hey.. there can only be one .. QUARTEX! And we bring you fire.
Ok. Lets do this!
Amibian + Smart pascal = A new beginning
In the Amiga community there is a sub-group of people with an agenda. They hoard and collect every piece of hardware they can get their hand on – then sell them at absurd prices on ebay to enrich themselves. This is not only ruining the community at large, ensuring that ordinary users cannot get their hands on a plain, vanilla Amiga without having to fork out enough dough to buy a good car.
“We also have a working Facebook clone – but we’re not
going into competition with Mark Zuckerberg either”
Thankfully not everyone is like that. There are some very respected, very good people who buy old machines, restore them – and sell them at affordable prices. People that do this as a hobby or to make a little on the side. Nothing wrong with that. No, its people that try to sell you an A2000 for $3000, or that pimp out Vampire accelerator cards at 700€ a piece thats the problem.
As for the price sharks, well – this has to stop. And Gunnar’s Amibian distro has already given the Amiga scalpers a serious uppercut. Why buy an old Amiga when you can get a high-end A4000 experience with 4 times as much power for around $35? This is what Gunnar has made a reality – and he deserves a medal for his work!
And myself, Thomas and the others in our little band of brothers will pick up the fight and stand by Gunnar in his battle. A battle to make the Amiga affordable for ordinary human beings that love the platform.
Amiga as a service
Yesterday I did a little experiment. You know how Citrix and VMWare services work? In short, they are virtualization application servers. That means that they can create as many instances of Windows or Linux as they want – run your applications on it – and you can connect to your instance and use it. This is a big part of how cloud computing works.
While my little experiment is very humble, I am now streaming a WinUAE instance display from my basement server pc, just some old bucket of chips I use for debugging, directly into Amibian.js (!). It worked! I just created the world’s first Amiga application server. And it took me less than 30 minutes in Delphi !
Amibian, Amibian.js, appserver, what gives?
Let’s clear this up so you dont mix them:
Amibian. This is the original Linux distro made by Gunnar Kristjansson. It boots straight into UAE in full-screen. All it needs is a Raspberry PI, the Amiga rom files and your workbench or hard disk image. This is a purely native (machine code) solution.
Amibian.js – this is a JavaScript remake of AmigaOS that I’m building, with the look and feel of OS 4. It uses uae.js (also called SAE) to run 68k software directly in the browser. It is not a commercial product, but one of many demos that ship with Smart Mobile Studio. “Smart” is a compiler, editor and run-time library sold by The Smart Company AS. So Amibian.js is just a demo, just like Amos shipped with a ton of demo’s and Blitzbasic came with a whole disk full of examples.
Amiga application server (what I mentioned above) was just a 30 minute experiment I did yesterday after work. Again, just for fun.
I hope that clears up any confusion. Amibian.js is a purely JavaScript based desktop made in the spirit of the Amiga – it must not be confused with Amibian the Linux distro, which boots into UAE on a Raspberry PI. Nor is it an appserver – but rather, it can connect to an appserver if I want to.
Generation Amiga post on Amibian.js earlier when I added some look & feel
With a bit of work, and if everything works as expected (I don’t see why not), I will upload both source and binaries to github together with Amibian.js.
There is only one clause: It cannot be used, recreated, included or distributed by Cloanto. Sorry guys, but the ROMS belong to the people, and until you release those into public domain, you wont get access to anything we make. Nothing personal, but pimping out roms and even having audacity to fork UAE and sell it as your own? You should be ashamed of yourself.
Are you in competition with FriendOS?
This question has popped up a couple of times the past two weeks. So I want to address that head on.
I make a product called Smart Mobile Studio. I do that with a group of well-known developers, and we have done so for many years now. The preliminary ideas were presented on my blog during the winter 2009, early 2010 and we started working (and blogging) after that. Smart Mobile Studio and it’s language, Smart Pascal (see Wikipedia article), takes object pascal (like freepascal or Delphi) and compiles to JavaScript rather than machine code.
The Smart compiler is due for OS4 once i get the A1222 in my hands
One of the examples that has shipped with Smart Mobile Studio, and also been available through a library called QTX, is something called Quartex Media Desktop. Which is an example of a NAS server front-end, a kiosk front-end (ticket ordering, cash machines etc) or just an intranet desktop where you centralize media and files. It is also node.js powered to deal with the back-end filesystem. This is now called amibian.js.
In other words – it has nothing to do with Friend software labs at all. In fact, I didn’t even know Friend existed until they approached me a few weeks ago.
The Quartex Media desktop has been around for ages
Amibian.js is just an update of the Quartex Media Desktop example. It is not a commercial venture at all, but an example of how productive you can be with Smart pascal.
And it’s just one example out of more than a hundred that showcase different aspects of our run-time library. This example has been available since version 1.2 or 1.3 of Smart, so no, this is not me trying to reverse engineer FriendOS. Because I was doing this long before FriendOS even was presented. I have just added a windowing manager and made it look like OS4, which also happened before I had any contact with my buddies over at Friend Software Labs (why do you think they were interested in me).
Early 2017 Linux bootloader by Gunnar
So, am I in competition with Friend? NO! I have absolutely no ambition, aspiration or intent for anything of the sorts. And should you be in doubt then let me break it down for you:
- Hogne, Arne, David, Thomas, Francois and everyone at Friend Software Labs are friends of mine. I talk almost daily with David Pleasence who is a wonderful person and an inspiration for everyone who knows him.
- Normal people don’t sneak around stabbing friends in the back. Plain and simple. That is not how I was raised, and such behavior is completely unacceptable.
- Amibian.js is 110% pure Amiga oriented. The core of it has been a part of Smart for years now, and it has been freely available for anyone on google code and github.
- For every change we have made to the Smart RTL, the media desktop example has been updated to reflect this. But ultimately it’s just one out of countless examples. We also have a working Facebook clone – but we’re not going into competition with Mark Zuckerberg for that matter.
- People can invent the same things at the same time. Thats how reality works. There is a natural evolution of ideas, and great minds often think alike.
Why did you call it Amibian.js, it’s so confusing?
Well it’s a long story but to make it short. The first “boot into uae” thing was initially outlined by me (with the help of chips, the UAE4Arm maintainer). But I didn’t do it right because Linux has never really been my thing. So I just posted it on my retro-gaming blog and forgot all about it.
Gunnar picked this up and perfected it. He has worked weeks and months making Amibian into what it is today – together with Thomas, our spanish superhero /slash/ part-time dictator /slash/ minister of propaganda 🙂
We then started talking about making a new system. Not a new UAE, but something new and ground breaking. I proposed Smart Pascal, and we wondered how the Raspberry PI would run JavaScript performance wise. I then spent a couple of hours adding the icon layout grid and the windowing manager to our existing media desktop – and then fired up some HTML5 demos. Gunnar tested them under Chrome on the Raspberry PI — and voila, Amibian.js was born.
And that is all there is to it. No drama, no hidden agendas – and no conspiracy.
I should also add that I do not work at Friend Software Labs, but we have excellent communication and I’m sure we will combine our forces on more than one software title in the future.
On a personal note I have more than a few titles I would like to port to FriendOS. One of my best sellers of all time is an invoice and credit application – which will be re-written in Smart Pascal (its presently a mix of Delphi and C++ builder code). The same program is also due to Amiga OS 4.1 whenever I get my A1222 (looking at you Trevor *smile*).
Well, I hope that clears up any misunderstanding regarding these very separate but superficially related topics. Amibian.js will remain 100% Amiga focused – that has been and remains our goal.
Ode to our childhood
Amibian is and will always be, an ode to the people who gave us such a great childhood. People like David Pleasence who was the face of Commodore in europe. A man who embody the friendliness of the Amiga with his very being. Probably one of the warmest and kindest people I can think of.
Francois Lionet, author of Amos Basic. The man who made me a programmer and that I cannot thank enough. And I know I’m not alone about learning from him.
Mark Sibly, the author of BlitzBasic, the man who taught me all those assembler tricks. A man that deserves to go down in the history books as one of the best programmers in history.
And above all – the people who made the Amiga itself; giants like Jay Miner, Dave Haynie, Carl Sassenrath, Dave Needle, RJ Michal (forgive me for not listing all of you. Your contributions will never be forgotten).
That is what Amibian.js is all about.
Patents and greed may have killed the actual code. But we are free to implement whatever we like from scratch. And when I’m done – your patents will be worthless..
Ghost of xmas past, or better known as the folder where old projects and abandoned ideas end up
I think everyone has a folder where they stuff old units, test projects and well – all those ideas that seemed awesome at the time, but you either discovered that it was crap, not enough time, or just forgot all about it. It happens. What can I say.
Yeah, I have one of those as well. And I also have those duplicate folders. You know, that time when you decided to make backups for everything – but then you got a new PC and now you dont have a clue which version to keep? And when you finally sit down with beyondcompare to get this sorted, you have managed to add to both copies.
Well, needless to say I had a day I just went apeshit about a year ago and sorted everything. And whatever did not fit into my neat new code-folder(s) was mercilessly stuffed into the dark, obscure folder known only as “stuff” or “misc”.
My first basic compiler, awwwww how cute
Well its not all rubbish! I did find a special version of my kitchen-sink research IDE, the IDE I use when trying out new code for DWScript, PaxCompiler and FreePascal. I think it has 4 compilers in it or something, yet its under 30 megabytes in size! Pretty neat 🙂
Visual Basic to Smart Pascal, awww…. just want to snuggle it 🙂
It also features a unified file-system! You can mount packages, FTP folders, network paths or local folders – the IDE could not care less, it is completely abstracted from the actual filesystem and relates only to symbols and identifiers. The actual storage is dealt with by the filesource classes.
Package, ftp, socket, local folder – the IDE could not care less
The Basic dialect here is essentially classical Visual Basic. The way you would write classes and deal with instances before dot net came along and forced all languages to look the same. I kinda like old visual basic, it had soul. It was completely useless except for scripting, and Delphi made it look like a joke – but as far as basic dialects go, it’s probably one of the cleanest and easiest to read.
DWScript, QTX Pascal (a custom fork of DWScript with a lot of cool features, some that Eric has added afterwards), Basic – and even freepascal! Thats the mother of all kitchen sinks!
The loss of QTX Pascal is a bit sad. I spent a couple of months doing my own fork of DWScript. Most of the features I added have now been included (although in another form) by Eric. But the codegen produced faster javascript. One of the things I spent some time on was optimization. Like getting rid of “variable = 0” type initialization if an assignment followed. I also added BeforeDestruction() and AfterConstruction() calls. This made an RTL a lot easier to write, but also more overhead. I was about to do conditional testing (so these would only be called if you actually used them) when I had to stop and work on Smart Mobile Studio again.
N++, exotic and functional!
This was one of my favorite language research projects. And dare I say, it is way ahead of it’s time. The idea is simple: with the future of computing being distributed, cloud based and powered by a multitude of computing modules from the US to South Africa — what would a programming language look like if built for 100% asyncronous, distributed execution?
Consider for instance, the execution of a procedure. Simple right? Well, in a truly distributed system – that procedure could execute anywhere. The whole point here is to utilize the combined power of each module (pc); which means a tasks could execute on different computers. A procedure could also be split up by the runtime environment and once again, be executed all over the place – with the dispatcher keeping track of when it’s done and whatever the result was (if any).
Only a spesific and generated on-demand “runtime context” would follow the task. This would contain things like variables it needs to execute, public symbols, basicaly just enough for the code to run successfully. This also includes resource forks (a term i used for accessing resources regardless of where they may be on the network).
The language design was actually done using Smart Mobile Studio (you can read more about the initial ideas here), and the first test modules ran on node.js (so no fancy graphics available sadly).
But it was incredibly fun to play with! But also time consuming to design a language for an execution model that havent really been invented yet. You can read more about some of my ideas for the execution model here.
I dont even think this execution model is out of the MIT labs yet — but it will once quantum compute models become commercially available (click here for a Berkley University published introduction to the basic computational models used in quantum information theory).
An N++ procedure (actually a microservice that consumed a WSDL service and calls it) looks like this:
program("service_test") {
handshake {
input {
/* Consume WSDL Web-Service Endpoint */
service1 @ service[URI:"http://www.test.com/SOAP/myService/WSDL/",
serviceType:SoapService];
}
output {
myProcess @ process("self");
stdio @ pipe("stdout");
}
}
/* Execute RPC call */
execute (stdio,service1) {
stdio:writeln("Calling webservice");
execute (*) {
var int32 result = 0;
set result = service1:getUserId("quartex","secret");
stdio:writelnF("UserID on server={0}", result);
} fail (e) {
stdio.writelnF("Calling soap service failed: {0}",e);
proceed;
}
}
/* Exit code for process */
set myProcess:exitCode = 0;
}
RML, Reduced markup language parser and runtime
Dont you just hate it when you lose the source-file for a programming language you made in a day, so now you dont have a clue what the syntax really was? I know, I know, but I honestly dont have the syntax or examples. It does with a disk ages ago.
RML was written to simplify generating HTML documents. There are a few things that you can optimize when it comes to HTML. First of all you know that the browser will read the document from top and downwards. This might not mean much at first glance, but it actually means that a language doesnt need the abillity to call emitter-code that has already executed (above). Think of it as being able to omit calls to procedure behind you in the source. That would be a mess for a language like delphi – but it doesnt impact a language that execute linear without any forms of jumps!
I seem to remember it looked something like this:
createobject(TRMLObject)
{
writestr("testing");
if(a=12)
{
readstr("mer testing");
}
}
createobject(TRMLHeading)
{
settitle("testing second object");
if(a=12)
{
writestr("heading object successfully done");
}
}
createobject(TRMLObject)
{
writestr("testing third object");
if(a=12)
{
readstr("mer testing from da third object");
}
}
The idea is ofcourse that you have some pre-defined objects, a bit like custom-controls in Smart Pascal, and you create and populate these in a top-down fashion.
Once defined, generating quite advanced HTML documents can be done extremely fast. Much faster than ASP, DWS or php since no jumps are involved.
Here is how you compile and run a small debug session (drop a memo and button on a form first):
var
FContext: TRMLParseContext;
FText: String;
begin
FText:= 'createobject(TRMLObject)'
+ '{'
+ ' writestr("testing");'
+ ' if(a=12)'
+ ' {'
+ ' readstr("mer testing");'
+ ' }'
+ '}'
+ 'createobject(TRMLHeading)'
+ '{'
+ ' settitle("testing second object");'
+ ' if(a=12)'
+ ' {'
+ ' writestr("heading object successfully done");'
+ ' }'
+ '}'
+ 'createobject(TRMLObject)'
+ '{'
+ ' writestr("testing third object");'
+ ' if(a=12)'
+ ' {'
+ ' readstr("mer testing from da third object");'
+ ' }'
+ '}';
// double the code, just to get some payload
FText := FText + FText;
memo1.text:='';
If RMLAllocContext(FContext,FText,[TRMLObject,TRMLHeading]) then
Begin
try
if RMLCompile(FContext) then
Begin
caption:='Compilation was a success';
If RMLExecute(FContext) then
Begin
Caption:='Executed successfully';
memo1.text:=FContext.pcOutput;
end;
end else
caption:=Format('compilation failed [%s]',[FContext.pcError]);
finally
RMLDisposeContext(FContext);
end;
end;
And here is the full code for the parser and runtime (still a few tidbits to work on, but its childs play). If you can make sense of it, knock yourself out 🙂
unit rml;
interface
uses sysutils, classes, dialogs;
type
TRMLObject = Class;
TRMLClass = Class of TRMLObject;
TRMLOperator = (opLess,opMore,opEquals,opNotEquals);
TRMLDataType = (daString,daNumber,daBoolean,daCustom);
TRMLEntryType = (etAssignment,etMethod,etRepeat,etCreate,etIf);
TRMLObjProc = Function (Const EntryData:Pointer):Boolean of Object;
(* represents a single condition in an IF statement *)
PRMLCondition = ^TRMLCondition;
TRMLCondition = Packed Record
coSource: String;
coTarget: String;
coOperator: TRMLOperator;
End;
(* represents a parsed code declaration *)
PRMLEntryDeclaration = ^TRMLEntryDeclaration;
TRMLEntryDeclaration = Packed Record
cxToken: String;
cxCondition: String;
End;
(* represents a compiled parameter *)
PRMLParameter = ^TRMLParameter;
TRMLParameter = Packed Record
prType: TRMLDataType;
prValue: Pointer;
prSize: Integer;
End;
(* represents a compiled code entry *)
PRMLEntryData = ^TRMLEntryData;
TRMLEntryData = Packed Record
edtype: TRMLEntryType;
edDeclaration: TRMLEntryDeclaration;
edObject: TRMLObject;
edParent: PRMLEntryData;
edMethod: TRMLObjProc;
edContext: Pointer;
edConditions: Array of PRMLCondition;
edParameters: Array of PRMLParameter;
edSubEntries: Array of PRMLEntryData;
End;
PRMLParseContext = ^TRMLParseContext;
TRMLParseContext = Packed Record
pcSignature: Integer;
pcCol: Integer;
pcRow: Integer;
pcPos: Integer;
pcLen: Integer;
pcData: String;
pcError: String;
pcOutput: String;
pcRoot: TRMLEntryData;
pcClasses: Array of TRMLClass;
End;
TRMLReadProc = Function (Var OutData;var Bytes:Integer):Boolean of Object;
TRMLWriteProc = Function (Var InData;Const Bytes:Integer):Boolean of Object;
TRMLProcEntry = Function (Const Entry:PRMLEntryData):Boolean of Object;
PRMLObjectIndex = ^TRMLObjectIndex;
TRMLObjectIndex = Packed Record
oiMethods: Array of record
omName: String;
omSyntax: Array of TRMLDataType;
omEntry: TRMLProcEntry;
end;
oiProperties: Array of record
oiName: String;
oiRead: TRMLReadProc;
oiWrite: TRMLWriteProc;
oiType: TRMLDataType;
end;
End;
TRMLObject = Class(TObject)
Private
FIndexData: TRMLObjectIndex;
FIndexPTR: PRMLObjectIndex;
Private
Function DoWriteStr(Const Entry:PRMLEntryData):Boolean;
Function DoReadStr(Const Entry:PRMLEntryData):Boolean;
protected
Procedure Output(Const Context:PRMLParseContext;Const Value:String);
Procedure RegisterProperty(Const Name:String;Const DataType:TRMLDataType;
Const _Read:TRMLReadProc;Const _Write:TRMLWriteProc);
Procedure RegisterMethod(Const Name:String;
Const Syntax: Array of TRMLDataType;
Const Entry: TRMLProcEntry);
Public
Property ObjectIndex:PRMLObjectIndex read FIndexPTR;
Constructor Create;virtual;
End;
TRMLHeading = Class(TRMLObject)
Private
FTitle: String;
Function DoSetTitle(Const Entry:PRMLEntryData):Boolean;
Function DoReadTitle(Var OutData;var Bytes:Integer):Boolean;
Function DoWriteTitle(Var InData;Const Bytes:Integer):Boolean;
Public
property Title:String read FTitle write FTitle;
Constructor Create;override;
End;
Function RMLAllocContext(var Context:TRMLParseContext;
Const Source:String;Const ClsBase:Array of TRMLClass):Boolean;
Function RMLDisposeContext(Var Context:TRMLParseContext):Boolean;
Function RMLCompile(Var Context:TRMLParseContext):Boolean;
Function RMLExecute(Const Context:TRMLParseContext):Boolean;
Function RMLParseEntry(Value:String;
var Declaration:TRMLEntryDeclaration;var Error:String):Boolean;
implementation
//###########################################################################
// TRMLHeading
//###########################################################################
Constructor TRMLHeading.Create;
Begin
inherited;
RegisterMethod('settitle',[daString],DoSetTitle);
RegisterProperty('title',daString,DoReadTitle,DoWriteTitle);
end;
Function TRMLHeading.DoSetTitle(Const Entry:PRMLEntryData):Boolean;
var
FTemp: String;
Begin
result:=length(Entry^.edParameters)>0;
If result and (Entry^.edParameters[0].prType=daString) then
DoWriteTitle(Entry^.edParameters[0].prValue^,
Entry^.edParameters[0].prSize);
end;
Function TRMLHeading.DoReadTitle(Var OutData;var Bytes:Integer):Boolean;
Begin
Bytes:=Length(FTitle);
If Bytes>0 then
move(FTitle[1],outData,Bytes);
end;
Function TRMLHeading.DoWriteTitle(Var InData;Const Bytes:Integer):Boolean;
Begin
SetLength(FTitle,bytes);
if Bytes>0 then
move(inData,FTitle[1],Bytes);
end;
//###########################################################################
// TRMLObject
//###########################################################################
Constructor TRMLObject.Create;
begin
inherited;
FIndexPTR:=@FIndexData;
RegisterMethod('writestr',[daString],DoWriteStr);
RegisterMethod('readstr',[daString],DoReadStr);
end;
Procedure TRMLObject.Output(Const Context:PRMLParseContext;
Const Value:String);
Begin
Context^.pcOutput:=Context^.pcOutput + Value;
end;
Function TRMLObject.DoReadStr(Const Entry:PRMLEntryData):Boolean;
var
FText: String;
Begin
result:=True;
With Entry^ do
Begin
FText:='Token: ' + edDeclaration.cxToken + #13#10;
FText:=FText + 'Condition: ' + edDeclaration.cxCondition + #13#10;
FText:=FText + #13#10;
Output(edContext,FText);
end;
end;
Function TRMLObject.DoWriteStr(Const Entry:PRMLEntryData):Boolean;
var
FText: String;
Begin
result:=True;
With Entry^ do
Begin
FText:='Token: ' + edDeclaration.cxToken + #13#10;
FText:=FText + 'Condition: ' + edDeclaration.cxCondition + #13#10;
FText:=FText + #13#10;
Output(edContext,FText);
end;
end;
Procedure TRMLObject.RegisterProperty(Const Name:String;
Const DataType:TRMLDataType;
Const _Read:TRMLReadProc;Const _Write:TRMLWriteProc);
var
FCount: Integer;
Begin
//FCount:=high(FIndexData.oiProperties) - Low(FIndexData.oiProperties) + 1;
FCount:=Length(FIndexData.oiProperties);
SetLength(FIndexData.oiProperties,FCount+1);
FIndexData.oiProperties[FCount].oiName:=Name;
FIndexData.oiProperties[FCount].oiRead:=_Read;
FIndexData.oiProperties[FCount].oiWrite:=_Write;
FIndexData.oiProperties[FCount].oiType:=DataType;
end;
Procedure TRMLObject.RegisterMethod(Const Name:String;
Const Syntax: Array of TRMLDataType;
Const Entry: TRMLProcEntry);
var
FCount: Integer;
FTemp: Integer;
Begin
//FCount:=high(FIndexData.oiMethods) - Low(FIndexData.oiMethods) + 1;
FCount:=Length(FIndexData.oiMethods);
SetLength(FIndexData.oiMethods,FCount+1);
FIndexData.oiMethods[FCount].omName:=Name;
FIndexData.oiMethods[FCount].omEntry:=Entry;
//FTemp:=high(Syntax) - Low(Syntax) + 1;
FTemp:=Length(Syntax);
If FTemp>0 then
Begin
SetLength(FIndexData.oiMethods[FCount].omSyntax,FTemp);
for FTemp:=low(syntax) to high(syntax) do
FIndexData.oiMethods[FCount].omSyntax[FTemp]:=Syntax[FTemp];
end;
end;
//###########################################################################
// RML util methods
//###########################################################################
Function RMLContainChars(Const Value:String;const Chars:String):Boolean;
var
x: Integer;
Begin
result:=True;
for x:=1 to length(chars) do
Begin
if pos(chars[x],Value)<1 then
Begin
result:=False;
Break;
end;
end;
end;
Function RMLScanFor(const Value:String;Const Target:CHAR;
Const Breakers:String;var Len:Integer):Boolean;
var
xpos: Integer;
Begin
result:=False;
Len:=-1;
xpos:=1;
while xpos<=Length(Value) do Begin If Value[xpos]=Target then Begin Len:=xpos-1; Result:=True; Break; end else Begin if pos(Value[xpos],Breakers)>0 then
Break;
end;
inc(xpos);
end;
end;
Function RMLisNumber(Const Value:String):Boolean;
const
CHARSET = '0123456789';
var
x: Integer;
Begin
Result:=True;
for x:=1 to length(Value) do
Begin
if pos(Value[x],CHARSET)<1 then
Begin
result:=False;
Break;
end;
end;
end;
Function RMLIsBoolean(Const Value:String):Boolean;
var
FTemp: String;
Begin
FTemp:=lowercase(trim(Value));
result:=(FTemp='false') or (FTemp='true');
end;
Function RMLisString(Const Value:String):Boolean;
var
x: Integer;
FLeft: Integer;
Begin
result:=False;
FLeft:=0;
(* check left side *)
for x:=1 to length(Value) do
Begin
if Value[x]='"' then
Begin
FLeft:=x;
Result:=True;
Break;
end else
if Value[x]<>#32 then
Break;
end;
(* check right side *)
If result then
Begin
for x:=Length(Value) downto 1 do
Begin
if Value[x]='"' then
Begin
If x>FLeft then
Break else
Begin
Result:=False;
Break;
end;
end else
if Value[x]<>#32 then
Break;
end;
end;
end;
Function RMLParseEntry(Value:String;
var Declaration:TRMLEntryDeclaration;
var Error:String):Boolean;
var
xpos: Integer;
Begin
fillchar(Declaration,SizeOf(Declaration),0);
Result:=RMLContainChars(value,'()');
if Result then
Begin
Result:=RMLScanFor(value,'(',')',xpos);
if result then
Begin
Declaration.cxToken:=trim(copy(value,1,xpos));
delete(value,1,xpos+1);
Result:=RMLScanFor(value,')','(',xpos);
if result then
Begin
Value:=TrimRight(Value);
Result:=xpos=(length(value)-1);
if result then
Declaration.cxCondition:=trim(Copy(Value,1,xpos));
end;
end;
end;
If not Result then
Error:='Invalid entry <' + value + '>';
end;
Function RMLAllocContext(var Context:TRMLParseContext;
Const Source:String;Const ClsBase:Array of TRMLClass):Boolean;
var
FCount: Integer;
Begin
If Context.pcSignature=SizeOf(Context) then
RMLDisposeContext(Context);
fillchar(Context,SizeOf(Context),#0);
Context.pcSignature:=SizeOf(Context);
Context.pcLen:=Length(Source);
Context.pcData:=Source;
FCount:=High(clsBase) - low(clsBase)+1;
If FCount>0 then
Begin
SetLength(Context.pcClasses,FCount);
for FCount:=Low(clsBase) to high(clsBase) do
Context.pcClasses[FCount]:=clsBase[FCount];
end;
result:=True;
end;
Procedure RMLDisposeEntryData(Const Data:PRMLEntryData);
var
FTemp: Integer;
x: Integer;
Begin
(* dispose of condition data *)
FTemp:=length(Data^.edConditions);
While FTemp>0 do
Begin
Dispose(Data^.edConditions[FTemp-1]);
dec(FTemp);
end;
SetLength(Data^.edConditions,0);
(* dispose of parameter data *)
FTemp:=length(Data^.edParameters);
While FTemp>0 do
Begin
If length(Data^.edParameters)>0 then
Begin
for x:=Low(Data^.edParameters) to high(Data^.edParameters) do
If Data^.edParameters[x]^.prSize>0 then
FreeMem(Data^.edParameters[x]^.prValue);
end;
Dispose(Data^.edParameters[FTemp-1]);
dec(FTemp);
end;
SetLength(Data^.edParameters,0);
(* dispose of sub entries *)
//Ftemp:=High(Data^.edSubEntries)-Low(Data^.edSubEntries)+1;
FTemp:=Length(Data^.edSubEntries);
While FTemp>0 do
Begin
RMLDisposeEntryData(Data^.edSubEntries[FTemp-1]);
Dec(FTemp);
end;
SetLength(Data^.edSubEntries,0);
(* dispose of script object *)
If not (Data^.edtype in [etIf,etRepeat,etMethod])
and (Data^.edObject<>NIL) then
Data^.edObject.free;
(* dispose of entry *)
Dispose(Data);
end;
Function RMLDisposeContext(Var Context:TRMLParseContext):Boolean;
var
FCount: Integer;
Begin
Result:=Context.pcSignature=SizeOf(Context);
If Result then
Begin
Context.pcSignature:=0;
Context.pcData:='';
Context.pcError:='';
FCount:=Length(Context.pcRoot.edSubEntries);
//FCount:=High(Context.pcRoot.edSubEntries)
//- Low(Context.pcRoot.edSubEntries) + 1;
While FCount>0 do
Begin
RMLDisposeEntryData(Context.pcRoot.edSubEntries[FCount-1]);
dec(FCount);
end;
SetLength(Context.pcRoot.edSubEntries,0);
end;
end;
//###########################################################################
// RML core methods
//###########################################################################
Function RMLImplements(Const MethodName:String;
Const obj:TRMLObject):Boolean;
var
FTable: PRMLObjectIndex;
FCount: Integer;
Begin
Result:=Obj<>NIL;
If result then
Begin
(* get object inex *)
FTable:=Obj.ObjectIndex;
//FCount:=High(FTable^.oiMethods) - low(FTable^.oiMethods) + 1;
FCount:=Length(FTable^.oiMethods);
Result:=FCount>0;
If Result then
Begin
for FCount:=low(FTable^.oiMethods) to high(FTable^.oiMethods) do
Begin
Result:=FTable^.oiMethods[FCount].omName=MethodName;
If Result then
Break;
end;
end;
end;
end;
Function RMLGetMethodEntry(Const MethodName:String;
Const obj:TRMLObject;var outEntry:TRMLObjProc):Boolean;
var
FTable: PRMLObjectIndex;
FCount: Integer;
Begin
Result:=Obj<>NIL;
If result then
Begin
(* get object inex *)
FTable:=Obj.ObjectIndex;
//FCount:=High(FTable^.oiMethods) - low(FTable^.oiMethods) + 1;
FCount:=Length(FTable^.oiMethods);
Result:=FCount>0;
If Result then
Begin
for FCount:=low(FTable^.oiMethods) to high(FTable^.oiMethods) do
Begin
Result:=FTable^.oiMethods[FCount].omName=MethodName;
If Result then
Begin
outEntry:=TRMLObjProc(FTable^.oiMethods[FCount].omEntry);
Break;
end;
end;
end;
end;
end;
Function RMLCreateObject(var Context:TRMLParseContext;
Const Objname:String;var outObject:TRMLObject;
var Error:String):Boolean;
var
FCount: Integer;
Begin
FCount:=High(Context.pcClasses) - Low(Context.pcClasses) + 1;
Result:=FCount>0;
if Result then
Begin
For FCount:=Low(Context.pcClasses) to high(Context.pcClasses) do
Begin
Result:=lowercase(Context.pcClasses[FCount].ClassName)=lowercase(objName);
If result then
Begin
outObject:=Context.pcClasses[FCount].Create;
Break;
end;
end;
end;
If not Result then
Error:='Unknown class <' + Objname + '>';
end;
Function RMLAddEntry(var Context:TRMLParseContext;
Var Declaration:TRMLEntryDeclaration;
Root:PRMLEntryData;var NewEntry:PRMLEntryData;
var Error:String):Boolean;
var
FCount: Integer;
x: Integer;
FTemp: String;
FLen: Integer;
FPar: PRMLParameter;
Begin
Result:=Root<>NIL;
If result then
Begin
(* create new entry *)
new(NewEntry);
(* Reset entry record *)
NewEntry^.edType:=etAssignment;
NewEntry^.edObject:=NIL;
NewEntry^.edMethod:=NIL;
SetLength(NewEntry^.edConditions,0);
SetLength(NewEntry^.edParameters,0);
SetLength(NewEntry^.edSubEntries,0);
(* Set basic values *)
NewEntry^.edParent:=Root;
NewEntry^.edContext:=@Context;
newEntry^.edDeclaration:=Declaration;
(* insert entry into parent *)
FCount:=Length(Root^.edSubEntries);
SetLength(Root^.edSubEntries,FCount+1);
Root^.edSubEntries[FCount]:=NewEntry;
(* tokenize *)
If declaration.cxToken='createobject' then
Begin
NewEntry^.edtype:=etCreate;
Result:=RMLCreateObject
(
Context,declaration.cxCondition,
NewEntry^.edObject,Error
);
end else
if declaration.cxToken='if' then
Begin
NewEntry^.edtype:=etIF;
NewEntry^.edObject:=NewEntry^.edParent.edObject;
end else
if declaration.cxToken='repeat' then
NewEntry^.edtype:=etRepeat else
Begin
(* method call? Make sure entry object supports this *)
Result:=NewEntry^.edParent.edObject<>NIL;
If Result then
Begin
(* check if object supports the method name *)
Result:=RMLImplements(declaration.cxToken,NewEntry^.edParent.edObject);
If Result then
Begin
(* Query object for method entry *)
Result:=RMLGetMethodEntry
(
declaration.cxToken,
NewEntry^.edParent.edObject,
NewEntry^.edMethod
);
If result then
Begin
NewEntry^.edtype:=etMethod;
NewEntry^.edObject:=NewEntry^.edParent.edObject;
(* now parse the parameter conditions *)
x:=0;
While x<Length(declaration.cxCondition) do Begin inc(x); If (declaration.cxCondition[x]=',') or (x=Length(declaration.cxCondition)) then Begin If x=Length(declaration.cxCondition) then FTemp:=FTemp + declaration.cxCondition[x]; FTemp:=trim(FTemp); If length(FTemp)>0 then
Begin
(* create a new parameter *)
FLen:=length(NewEntry^.edParameters);
setlength(NewEntry^.edParameters,FLen+1);
New(FPar);
If RMLIsString(FTemp) then
FPar^.prType:=daString else
if RMLIsNumber(FTemp) then
FPar^.prType:=daNumber else
if RMLIsBoolean(FTemp) then
FPar^.prType:=daBoolean else
FPar^.prType:=daCustom;
Case FPar^.prType of
daString:
Begin
Delete(FTemp,1,1);
Delete(FTemp,length(FTemp),1);
FPar^.prSize:=Length(FTemp);
FPar^.prValue:=AllocMem(FPar^.prSize);
move(FTemp[1],FPar^.prValue^,FPar^.prSize);
end;
daNumber:
Begin
FPar^.prSize:=SizeOf(Integer);
FPar^.prValue:=AllocMem(FPar^.prSize);
PInteger(FPar^.prValue)^:=StrToInt(FTemp);
end;
daBoolean:
Begin
end;
daCustom:
Begin
end;
end;
NewEntry^.edParameters[FLen]:=FPar;
FTemp:='';
end else
Begin
//Invalid parameter error
end;
end else
FTemp:=FTemp + declaration.cxCondition[x]
end;
{
Validate parameter datatypes here!
If (Length(NewEntry^.edParameters)>0) then
Begin
for x:=Low(NewEntry^.edParameters) to
high(NewEntry^.edParameters) do
Begin
newEntry^.edObject.
end;
end; }
end;
end else
Begin
// property assignment test here
end;
end;
end;
(* Failed to tokenize? *)
If not Result then
Begin
(* dispose of entry data *)
Dispose(NewEntry);
NewEntry:=NIL;
SetLength(Root^.edSubEntries,FCount);
Context.pcError:=Format('Invalid token "%s"',[declaration.cxToken]);
end;
end else
Error:='AddEntry failed, root can not be NIL';
end;
Function RMLParseObject(Var Context:TRMLParseContext;
Const Root:PRMLEntryData):Boolean;
var
FChar: Char;
FTemp: String;
FDeclaration: TRMLEntryDeclaration;
FNewEntry: PRMLEntryData;
Begin
Result:=Context.pcSignature=SizeOf(Context);
If result then
Begin
(* update cursor *)
inc(Context.pcPos);
inc(Context.pcCol);
while Result and (Context.pcPos<Context.pcLen) do
Begin
FChar:=Context.pcData[Context.pcPos];
Case FCHAR of
#13:
Begin
inc(Context.pcRow);
inc(Context.pcPos);
Context.pcCol:=0;
Continue;
end;
';':
Begin
Result:=RMLParseEntry(trim(FTemp),FDeclaration,Context.pcError);
If result then
Result:=RMLAddEntry
(
Context,
FDeclaration,
root,
FNewEntry,
Context.pcError
);
If Result then
Begin
inc(Context.pcPos);
inc(Context.pcCol);
If FNewEntry^.edtype=etIF then
Result:=RMLParseObject(Context,FNewEntry);
end;
FTemp:='';
end;
'{':
Begin
Result:=RMLParseEntry(FTemp,FDeclaration,Context.pcError);
If Result then
Begin
Result:=FDeclaration.cxToken='if';
If result then
Begin
Result:=RMLAddEntry
(
Context,
FDeclaration,
Root,
FNewEntry,
Context.pcError
);
If Result then
Result:=RMLParseObject(Context,FNewEntry);
end;
end;
FTemp:='';
end;
'}':
Begin
inc(Context.pcCol);
inc(Context.pcPos);
Break;
end;
else
Begin
FTemp:=FTemp + FChar;
inc(Context.pcCol);
inc(Context.pcPos);
end;
end;
end;
end;
end;
Function RMLExecute(Const Context:TRMLParseContext):Boolean;
Function RunEntry(Const Item:PRMLEntryData):Boolean;
var
FSubCount: Integer;
x: Integer;
Begin
result:=True;
Case Item^.edtype of
etCreate,
etAssignment:
Begin
FSubCount:=Length(Item^.edSubEntries);
for x:=1 to FSubCount do
Begin
result:=RunEntry(Item^.edSubEntries[x-1]);
If not result then
break;
end;
end;
etMethod:
Begin
{result:=RMLGetMethodEntry(Item^.edDeclaration.cxToken,
Item^.edObject,FEntry);
If result then
result:=FEntry(Item); }
Result:=TRMLProcEntry(Item^.edMethod)(Item);
end;
etRepeat:
Begin
//FSubCount:=Length(Item^.edSubEntries);
end;
etIf:
Begin
FSubCount:=Length(Item^.edSubEntries);
for x:=1 to FSubCount do
RunEntry(Item^.edSubEntries[x-1]);
end;
end;
End;
Begin
Result:=Context.pcSignature=SizeOf(Context);
If result then
Begin
result:=length(Context.pcError)<1;
If result then
result:=RunEntry(@Context.pcRoot);
end;
end;
Function RMLCompile(Var Context:TRMLParseContext):Boolean;
var
FChar: Char;
FTemp: String;
FDeclaration: TRMLEntryDeclaration;
FNewEntry: PRMLEntryData;
Begin
Result:=Context.pcSignature=SizeOf(Context);
If result then
Begin
Context.pcCol:=0;
Context.pcRow:=0;
Context.pcPos:=1;
Context.pcError:='';
while Result and (Context.pcPos<Context.pcLen) do
Begin
FChar:=Context.pcData[Context.pcPos];
Case FCHAR of
#13:
Begin
inc(Context.pcRow);
inc(Context.pcPos);
Context.pcCol:=0;
Continue;
end;
'{':
Begin
Result:=RMLParseEntry(FTemp,FDeclaration,Context.pcError);
If Result then
Begin
Result:=FDeclaration.cxToken='createobject';
If result then
Begin
Result:=RMLAddEntry
(
Context,
FDeclaration,
@Context.pcRoot,
FNewEntry,
Context.pcError
);
If Result then
Result:=RMLParseObject(Context,FNewEntry);
end;
end;
FTemp:='';
end;
'}':
Begin
end;
else
Begin
FTemp:=FTemp + FChar;
inc(Context.pcCol);
inc(Context.pcPos);
end;
end;
end;
end;
end;
end.
Why C# coders should shut up about Delphi
EDIT: Some 3 years after this satire post was published, a website called “.NetRocks” decided to make a number out of it. The satire clearly went over their heads, and the facts I outline in the post was met with ridicule. I’m a bit disappointed with .NetRocks, because this could have been a great opportunity for them to learn more about modern object-pascal development and it’s ecosystems. They seem genuinely baffled that Delphi and Object-Pascal in general was being used in 2016 (and 2019), and their behavior and response demonstrates the very arrogance and ignorance we have come to expect from “brogrammers” of their type.
I must admit that I’m somewhat shocked that a massive show like .NetRocks is utterly ignorant of the fact that Object-Pascal as a language has millions of users around the world. You have compilers like Oxygene from RemObjects that targets .Net, Java, x86, ARM and WebAssembly. So their whole podcast on “how to port Delphi applications to C#” is absurd, because that’s not how you migrate code from Delphi to .Net. The Elements ecosystem, which covers C#, Pascal, Java, Swift and GoLang installs straight into Visual Studio and adds Object-Pascal to the environment. You can also convert source-code between Pascal and C# through automation. Manually going about such tasks is amateurish to say the least, but I honestly didn’t expect much when they open their podcast with verbal abuse.
While .NetRocks is manually porting pascal to C#, the rest of us use the Elements compiler and just compile the code to .Net Assemblies. You can also convert the code to any of the 5 languages Elements support. I have to wonder what rock .NetRocks lives under to be oblivious to this
Another point .NetRocks seem to have missed, is that Delphi has been aggressively optimized since Embarcadero took over from Borland. In 2016 Object Pascal ranked as #14 [2.164 score] on the Tiobi index over most popular programming languages [globally], just nine points below C#. So the most cost effective way of getting your legacy Delphi application modernized, is to just buy Elements from RemObjects, or upgrade your Delphi version to this century.
If they had bothered to check they would also have learned that I manage some 16+ developer groups on social media, with roughly 50.000 developers in total. I have also worked for Embarcadero that makes Delphi, so my statements while whimsical and satirical, were based on facts. I work with Fortune 500 companies in the US, as well as large companies around Europe, that all rely on Object-Pascal for their daily operations. Heck, even the Walt Disney company wanted to get rid of .Net in favour of Object Pascal in 2018. Turns out JIT is a disaster when coding against hardware interrupts and “old school” automation. Who could have guessed? (that was satire btw).
Some people seem to think that Delphi ended with version 7 back in the 90s. Delphi has been under heavy development since Embarcadero took over for Borland. .NetRocks doesnt even mention that buying a modern version of Delphi is an option companies should consider. And if they absolutely have to migrate to C#, then Elements is what professionals use.
On the whole, It’s alarming that the guys at .NetRocks seem clueless to the tools professional developers use for specialized work. Porting a Delphi codebase by hand its, well, amateurish at best. They might as well do a podcast on how to use ms-paint to create art, because Adobe Photoshop or Illustrator is not what people use (that was satire btw).
I am also pleased however, because .NetRocks unwittingly demonstrated my point to perfection 🙂 Instead of learning more about Anders Hejlsberg’s journey and life before he joined Microsoft, how the .Net class hierarchy is heavily influenced by the Delphi VCL framework, or how the concepts that ended up as .Net were in fact prototyped and experimented with earlier — they jumped straight to ridicule. Of a satire post (lol!).
You can click here for my full reply. And seriously, how it’s possible to mistake this old post as anything but heavy satire is beyond me. But I am thankful to .NetRocks for demonstrating my point. I could not have asked for a better example of arrogance and ignorance 🙂 Rock on!
When in Rome
Those that follow my blog or know me personally – also know that I don’t go out of my way to be confrontational or disrespectful. I try my best to follow the middle-path, to see positive in all things. But sometimes you face a level of stupid that begs a response. A verbal one. And this is one of those cases.
Lately I see more and more Delphi developers getting into debates with C# developers, and they are confronted with an attitude and belief-system that quite frankly is utter nonsense. It’s not based on history or facts but disputes rooted in childish “isms”. A mix of old habits and an unhealthy obsession with the notion that new means better.
It is nothing short of an intellectual emergency.
Old is bad?
If that is the case then C/C++ should be utter shit, because C is 3 years older than Pascal. Pascal was created to replace C and get away from (amongst other things) the absurd and time-consuming header file practice, not to mention the recursive hell that still occur when headers reference other headers and conditional symbols pile up.
Do you know how curly brackets became popular? Jump into your nearest Tardis and set the destination to the mid 1960’s. Back when mainframes were the size of Portugal and 1024 bytes of memory was the bomb. You see, back then memory was sparse and pretty much all the languages (or toolkits) needed to save space. So the { } curly’s were used as tokens for words like “begin” and “end”. Why? Because one ascii byte is less than five ascii bytes. Every byte counts when you only have a few Kb of memory.
Ps: when I use the word toolkit here I really mean a random soup of asm snippets on a loom. The bones collected to make C was actually a mix of homebrew toolkits made by different engineering teams at the time. And each of these toolkits had their own fanclub /slash/ legal representation; because all of them, miraculously, had invented the exact same thing at the same time before everyone else. These guys kept on fighting over who owned what well into the Watcom era. It’s probably the longest standing demonstration of a grown-up tantrum since the roman emperor caligula outlawed clothing.
Fast forward to 1970 and the amount of memory on mainframes had already doubled a few times just like Moore had predicted five years earlier. The C standard that Dennis Ritchie had painstakingly wrangled from the cold, greedy hands of grumpy old technicians was due for an upgrade. Tests were done by very dull, very serious men in grey clothing – and in their horror they discovered that human beings would interact with words and phrases faster than symbols and glyphs. So a word like “begin” would be picked up by the brain faster than {. It is just knockout stuff isn’t it.
You cannot praise Anders as a genius architect and at the same time throw his life work in the thrash. Perhaps you should ask yourself why Anders loved object pascal so much that he dedicated half his career making sure universities had access to it.
Living on the edge, Nicolaus Wirth rocks the scene!
Enter Niklaus Wirth, a man so clever and academic that he doesn’t catch exceptions; He is the exception. But next to his colleagues over at Berkley our dear Niklaus is the proverbial rock-star. He decided that in order to make programming more accessible for students, easier to learn and above all, establishing a standard for safer code, that they should take science to heart and include that in a programming language design. So he put on his gravest lab coat and began to implement “a better C” where he applied the research that words are picked up by the brain faster than tokens and symbols. Nine months later pascal was born, and to celebrate Niklaus used colored socks that day. A risky bit of business that could have cost him his seat at the strangely brown sorority club, in which he was master of ceremonies. But hey, Nikolaus is a rebel and this was the swinging 70’s after all.
My point? This is why pascal uses “begin” and “end” as opposed to { and }. It’s one of the many aspects of the Pascal language that makes it so easy to master.
Whimsical satire aside: with the knowledge that C is actually older than pascal, not to mention that it is identical in features, depth and complexity to Pascal– what is it going to be? Do you still cling to the notion that older have to be of less value than new? Or could it in fact be that you haven’t really looked closer at object pascal to begin with? Have you even installed a modern version of Delphi and given it a worthy test drive? Or do you just speak out based on what you like and know, as opposed to what you don’t know and haven’t even tried.
My 8-year-old daughter is like that. She makes up her mind about food before she has even tasted it. Surely a trained engineer can do better?
And I say that because — if we follow your line of reasoning, C/C++ should be incapable of producing C#. How can something old and outdated possibly produce something new and exciting? You do know that C# is written in C and assembler right? And that Java and all the other high level languages out there didn’t spontaneously self assemble. They are all written in C, assembler or pascal. And there is a reason for that.
Just like nature have different levels, from particles to minerals, from minerals to bacteria, from bacteria to plants, from plants to insects (and so on) — so does a computer. Each of these levels have laws and principles that govern them, and no matter how much you want to introduce something from a higher level into a lower level – that’s just not how reality works.
The fact that I have to explain this in 2016 demonstrates how education and academia has degraded after C and Pascal was replaced with Java.
The law is very simple: each level can only emit complexity upwards. So assembler is used to produce C and pascal, C and pascal is used to produce C++ and object pascal, C++ and object pascal is used to bring about every other single piece of tech you take for granted.
So when you criticize Delphi but use C#, you are just underlying that you dont really understand the difference between archetypical languages and context based languages.
It’s like saying Visual Basic 5 is cooler than assembler. You don’t debate with people like that, you just look at them with pity. I don’t know how many years a software engineering degree is elsewhere, but if you just spent 3 years in college and 2 years at a university followed by one or two additional years getting your diploma – and this is the level of insight you graduate with then I pity you. And I pity your employer because sooner or later you will bring about the apocalypse.
If we apply rational logical thought to the proposition at hand, we cannot but conclude that the notion of “new is always better” is false. It is to mistake marketing for facts and capitalism for science. The fundamental principles of computing wont change because you favour a high-level programming language. The C/C++ embedded market wont magically roll over, because C# is a mitigated disaster on embedded devices. And I speak from experience, not preference.
Your teacher should have taught you this: “In a computer, like nature, the past is always alive“. Look closely at your brand new PC or Mac: Bios, Vesa screen modes, non linear memory, hardware interrupt vectors and a cpu that supports no less than 3 instruction sets. You have layer on layer of increasing complexity (a.k.a “the past is still there”). Why not call up Linus Torvalds and ask him why he’s not using C# in his line of work; or NVidia why their GPU pipeline only ships with C headers and not C# classes. I sure as hell wouldn’t want to be on the receiving end of that call.
The only C# compiler worth using, is actually RemObjects C# (the Elements compiler). Because unlike mono or ms’s compilers, Elements builds proper binaries on the same level as C/C++ and Pascal. You can actually write kernel modules with Elements if you like, something you cant do with Mono or Visual Studio. Heck, .net services still require a service host on Windows.
C# is better than Delphi?
Anders Hejlsberg, the father of many languages
Actually, it’s not. What today is known as .net, from its CIL intermediate language right down to the global assembly cache is pretty much Delphi with a new syntax parser on top. Yes you read that correctly the entire .net product family is practically Delphi refactored. The whole bytecode regime was one of the last skunkwork projects Anders worked on, and was actually described on Borland’s news-servers years before it magically re-appeared as Microsoft’s flagship product.
Anders Hejlsberg created and prototyped these technologies (as did many other companies, since Java made bytecodes cool) while he was working at Borland. And indeed, the father of C# and entire dot net stack is none other than the father of Delphi.
At Borland (in the borland.language newsfeed I vividly remember) these technologies went mentioned under the codename “portable Delphi”. I followed the thread on the news servers for quite some time. It was no secret that Borland was tinkering away on the next big thing, and I believe it was ultimately Borland’s response to Java. Because at the time Java had eaten quite a chunk of Borlands educational seat licenses. And had Anders continued working for Borland, Delphi and C++ builder would perhaps be bytecode based today.
Factoid: The first project Anders was assigned to when he joined Microsoft, was J#. Microsoft’s attempt at hijacking the Java language. Microsoft lost the legal battle and was forced to pivot. The end result would be C#.
So when you, the C# fanboy, propose that Delphi is somehow inferior to dot net, or that object pascal is outdated and technically behind C# you quite frankly have no clue what you are talking about. Is it generics? We have that. Is a rich and powerful RTTI? Got that covered. Anonymous procedures? Its there too. In fact, the whole misunderstanding here stems from the fact that C# developers generally think “Delphi 7” is what Delphi is all about. Which is the same that saying that .Net is what came out in version 1. Who the hell uses Delphi 7 in our day and age.
But just to make this crystal clear: Anders Hejlsberg is not just the father of C# and dot net: he is also the father of Delphi. And before that he gave birth to Turbo Pascal. In fact, Anders have 3 (if not four) Pascal development systems behind him before he came to Microsoft.
Borland, land of the free
Borland, the company Anders worked for, was put out to pasture by Microsoft. Bill Gates launched a financial onslaught that few companies on the planet could have endured. The reason was that Microsoft really had the worst developer tools on the market (Visual Basic and Visual C++), while Borland represented the very best. So for a long time Borland utterly demolished Microsoft when it came to software development.
When Borland later began to flirt with Linux (Delphi Kylix) it must have made the R&D department at Microsoft piss themselves. Delphi is responsible for thousands of desktop applications synonymous with Windows – and if Linux users got their hands on RAD tools like Delphi, with thousands of applications ready to be ported over? Do the math. You have to remember that this was before cloud computing. Microsoft didn’t have a fallback strategy and depended on desktop and server sales. Linux could do everything Windows could, but it lacked the desktop applications, the user friendliness and features Borland made simple.
Wishing for stuff doesn’t make it true, except in the movies
What Microsoft did was, basically, to buy out Anders (they also picked up a huge part of Borlands R&D department) from Borland and at the same time attack the company from every angle. Drowning them in bullshit lawsuits, patent claims and all the nasty, cowardly practices Microsoft was into back then (we all remember Netscape and how that went).
Bill Gates called Anders up personally and gave him “an offer I could not refuse“. Whatever that means.
So to put this myth to bed and be done with it: C# as a language and dot net as a technology did not originate with Microsoft. It has Borland written all over it and its architecture is intimately joined at the hip with Delphi and C++ builder. Companies like Microsoft dont innovate any more, they buy competence and re-brands it as their own.
Dot Net framework, you mean the VCL?
If that is not enough, take a long, hard look at how the .net framework is organized. Pay special attention to the features and methods of the base-classes and RTTI. What you are looking at is the VCL, the visual component library, Delphi and C++ builder’s run time library.
If you know Delphi and it’s RTL intimately as well as C#, you quickly recognize the architecture. The syntax might be different, but the organization and names are all too familiar.
There was never any secret where this came from, but it’s not advertised either so I don’t blame people for not recognizing it. But when people start to behave nasty over what should be either a passionate hobby or a professional discipline, then truth must be told. The very foundation of the dot net framework is taken straight out of the VCL.
And why shouldn’t it? The whole platform was whispered about under the codename “portable Delphi” to begin with, it was even designed by the same author – so why should it come as a surprise that it’s basically Delphi reincarnated with a different syntax parser? Either way a few bells should start ringing in your head – telling you that you may have been wrong about Delphi and object pascal in general. Probably because you know very little about Delphi to begin with.
So, what you perhaps believe sets C# apart from Delphi, what you imagine gives dot net an advantage over Delphi – is in reality the opposite: it is Delphi. It originated in Delphi, was inspired by Delphi and flows straight from the inventor of Delphi himself. Anders is responsible for two distinct object pascal development platforms prior to working for Microsoft. Take some time to reflect on that.
You cannot praise Anders as a genius architect and at the same time throw his life work in the thrash. Perhaps you should ask yourself why Anders loved object pascal so much that he dedicated half his career making sure universities had access to it.
Strength in diversity
The fact that C/C++ is alive and well today serves to prove my point: age has no meaning when dealing with fundamental technology. These languages embody the fundamental principles of computing. They don’t grow old because they represent a fundamental level that can never be replaced. There is now law that says C must be there, what persists is ultimately the features that C brings – spanning from assembly to the desktop. Those same features can be found only in one other language, namely Pascal.
Things are the way they are for a reason. And programmers before you established standards that address things you might not have to deal with, but that’s because they dealt with it before you. Just because you don’t see the plumbing doesn’t mean it’s not there.
These are the compilers responsible for the foundation on which all other languages, operating-systems, applications and services rest. Remove those, and the whole house of cards come crashing down.
So C/C++ and object pascal continue to exist, thrive even, precisely because they interface with the hardware itself, not a virtual machine or bytecode format. Delphi as a product is naturally tailored for operating-systems, as is C/C++. But that is because we use programming languages to create applications. There is nothing in the way of someone using compilers like freepascal and writing an operative system from scratch.
Ultibo is an ARM operative system written purely in object pascal
By contrast, C#, Java and Visual Basic are context dependent languages. They exist purely in context with an operative system or run-time environment. They can’t exist without these and lack the depth C and pascal brings to the table. The depth to span many levels, from the lowest all the way up to cloud and domain driven programming.
Just look at Android, an operative system written in Java right? Well that’s not really true. The Android operative system depends on a native bootstrap, a piece of code that establishes the run-time environment for Java applications. So when you fire up your Android phone, tablet or set-top box, the first thing to boot is a loading mechanism written in vanilla C. This in turn mounts a filesystem and continues to loads in bunch of native drivers. Drivers for the display, storage, wi-fi and all the hardware components that ultimately make up your SoC (system on a chip).
Its only after this long process that the environment is capable of hosting the Java virtual machine. Then and only then is the device capable of executing Java. This is why languages like Java is called context-based, because they run in context with an established system.
Notice the “C” part of the Java NDK? Wasnt that exactly what Java was supposed to solve? All that hype and here you are, back at where you started!
So Java, which for the past decade or more have bombarded us with advertising like “platform independence” is a total flop! Its like communism and looks good on paper, but once you start testing these ideas in real life – they just don’t work.
Turned out that the java virtual machine when running on small devices like phones and tablets, choked the CPU. Which in turn used more electricity to keep up, which in turn rendered the battery life of these devices null and void. Talk about irony. All that hype and Java couldn’t even deliver its most basic promise: platform independence. And you removed C and pascal from our universities for this? It’s a disaster en-par with the burning of the great library of Alexandria.
There is more than a drop of ego and arrogance in this material. Young people especially like to believe they are smarter than those who lived before them. It’s a very selfish idea that really need to be addressed. Take someone like Alan Turing for example, the inventor of what we call computers today. This man assembled and created the first computer by hand. He even hand carved the wheels that made the mechanism run (which was mechanical during the second world war). I would happily bet $1000 that Alan could out program the lot of us had he gotten the chance. He has been dead for many years, but to imagine yourself smarter or more capable than him would be the definition of arrogance.
PS: This is where we come into the realm of wisdom versus knowledge. The difference may be subtle, but it’s so important to distinguish between them.
C# would have been a sad little language without the work of this man, Miguel de icaza
Either way without the native context to support it Java turned out to be a blundering behemoth. And its the exact same with C#. This is why Miguel de Icaza work on Mono has been so instrumental to the success of dot net and C# in particular. It was Miguel and his team that pushed C# into native compilation versus JIT for mobile devices. C# would never have been allowed on IOS or Android otherwise.
So that is the difference between archetypical languages and context based languages. Hopefully you now see why this negative attitude towards Delphi and object pascal doesn’t make sense. It really is just as stupid as these cults that believe in dinosaurs and that human beings once lived side by side with a T-rex. Anyone with a highschool diploma should instinctively know better.
The problem is when people in great enough numbers start to believe in this nonsense, because then they can actually damage important mechanisms of our eco-system. Insane beliefs really is dangerous both socially and technically.
Grow up!
I have no problem using C# and consider it a valuable tool in my toolbox. Same with Delphi. But when it comes to kick-ass desktop applications for Windows and OS X, Delphi has no rival. It gives you a productive edge that I havent found in any other language. And I don’t base that on preference. I base that on engineering and the years of development invested in Delphi by both Borland and Embarcadero.
I just find it odd that C# developers seem to lack the ability to speak well about Delphi, because it’s a tell-tell sign of immaturity to be incapable of that. Object pascal and C++ is identical in every way (except multiple inheritance, which even C++ programmers shun like the plague so no loss there). Why then should educated programmers find it reasonable to praise C but thrash pascal? It’s like saying that one car is faster than another – but when you pop the hood they have the same engine. The exact same machine code generator even. It is psychological masturbation. We are coders, we shouldnt do things like that. We should examine the system carefully and then make up our minds.
And what is a programming language anyway? It is the ability to describe the solution to a problem. The ability to define and control how a series of activities execute. Each language represents a methodology for solving problems. A mindset. Nothing more.
Languages like C, C++ and object pascal will always exist, they will always thrive, because they are instrumental to everything else (desktop, operative system, services, everything that makes a computer capable of what it does). It doesn’t mean that Delphi is the solution to all problems, far from it. And that is ok.
Context based languages are more often than not designed for a purpose. Java was initially designed as a network language. This makes Java infinitely more suitable for dealing with that line of work. You can do the exact same in C but you would have to add a few libraries and establish a platform. Which is exactly what C and pascal is good at, implementing fundamental technology. Object pascal is a language you would use when creating other languages. C is more popular for virtual machine development, no doubt there, but there is nothing in C/C++ that you wont find in object pascal.
Interestingly, languages that many believe is long since extinct, like Cobol, are actually alive. Cobol was created to deal with large stacks of numbers and to push associated data through formulas. That is its context and no other language is better suited for the banking sector than Cobol. Which incidentally is the one place where Cobol is used to this day. And why not? Why waste millions trying to hammer in a nail with a hacksaw? Why not use a language especially designed for that purpose? SQL is likewise a language especially tailored for its tasks. I don’t hear anyone bitching about how old and outdated that is. It wont grow old because it embodies the principle of relational data.
And just what exactly is old about Delphi? Generics? Posix support? Weak and strong references? Garbage collection? Attributes? class, record and enum helpers? Dependency injection? Model view controller frameworks? The ability to write assembler side by side with pascal? The ability to link with C/C++ objects? Cross compilation for Windows, OS X and mobile platforms? What manner of logic are you using where you praise other languages for these features, but for Delphi the same features makes it old? Stop repeating what someone once told you like a mindless parrot and investigate. You are an engineer, a field of science; start acting like one.
In my work I use several languages: JavaScript, Delphi, Smart Pascal, Mono C# and even Python. I have also fallen in love with LUA believe it or not.
For web-pages I prefer Smart pascal which compiles to JavaScript (node.js is fantastic). Then you have shell scripting, a touch of x86 assembly code, a dash of bash on my Linux box and .. well, there is strength in diversity.
It all depends on what is best for the customer and what language yields the most robust solution. Sometimes C# is better suited for my clients infrastructure, other times Delphi makes more sense. But if I was to pick a single language to cover all of them, it would either be object pascal or C++ (gcc). These really have no rivals and will never go away.
These are the compilers responsible for the foundation on which all other languages, operating-systems, applications and services rest. Remove those, and the whole house of cards come crashing down.
So using many languages is actually an enriching experience. It brings more depth to our way of thinking and (consequently) affect the quality of our products.
Anyways, I believe congratulations are in order, because if your favorite language is C#, you have in fact been using Delphi without knowing it. There is so much Delphi in C# and they both originate from the same author. So why not pay a visit to C#’s older brother?
So go download Delphi, Lazarus and Remobjects Oxygene – enjoy the decades of research and excellence these dialects represent. It won’t hurt you, it will probably do you a great deal of good.
Smart Mobile Studio, datasets and websockets
It’s all about the bytes
If you have been playing with the beta you have probably noticed a few cool new units in the RTL. First there is system.memory, system.typecon, followed by system.streams. And last but not least: system.dataset.
What’s so special about these units? Well, they give you the same level of low-level (read: byte) access as you enjoy under Delphi. If you know your way around JavaScript you must have noticed that this is a sensitive subject, a really sensitive subject (!). JavaScript is hopeless when it comes to binary data, and there is no such thing as a “stream” in the browser.
You wont believe some of the solutions out there which pass for “top notch” JavaScript. Most of them involving cheap hacks like stuffing bin-hex’ed data into strings. This means you need a separate layer of encoding before you push the data to a server (or de-serialize it to JSON). Well we dont have that problem 🙂
So, with all these units firmly in place we can now enjoy the same level of power, simplicity and elegance as under native Delphi or freepascal. And this is where TW3Dataset comes in. Because it’s actually a fully OOP dataset which saves to a very simple binary format (!)
TW3Dataset
Fast, binary, pure object pascal
Let’s say you have a mobile solution which needs to store uniform data. It can be a phone-book, a list of high-scores for a game, time samples or your latest RSS feed poll. Traditionally you would sculpt it all using arrays and then serialize the whole shabam to JSON; or even worse: make use of WebSQL, PouchDB, TaffyDB or other JavaScript dependencies.
Well that is now a thing of the past. For simple in-memory dataset chores, TW3Dataset covers your basic needs more than adequate. It also saves to TStream, meaning that saving and loading from the new storage system (more about that later) is reduced to a “one liner”.
Is it hard to use?
Nope. In fact it’s even simpler to use than TClientDataset under Delphi (or more or less identical, except with no indexing just yet). Here is a short example of how to create a dataset, populate it and save it to a stream:
var
mDataset: TW3Dataset;
mData: TStream;
x: Integer;
begin
mDataset := TW3Dataset.create;
try
mDataset.FieldDefs.Add('id',ftAutoInc);
mDataset.FieldDefs.add('name',ftString);
mDataset.CreateDataset;
for x:=1 to 1000 do
begin
mDataset.append;
mDataset.Fields.FieldByName('name').AsString := 'Name #' + x.toString;
mDataset.Post;
end;
mData := TMemoryStream.create;
try
mDataset.SaveToStream(mData);
writeln('Dataset bytesize = ' + mData.size.toString);
finally
mData.free;
end;
finally
mDataset.free;
end;
As you can see from the code above some calculated field types are supported (Autoinc), and TW3Dataset is going to see some heavy expansion as it’s merged with my universal DB framework. Think FireDac but for the browser.
Other cool stuff
If TW3Dataset is to limited and you want SQL support, Smart Mobile Studio now ships with SQLite compiled for JavaScript. Yes you read right, SQlite has been compiled for JavaScript and reads, writes and generates identical files to the native SQLite versions. That’s pretty darn cool (although it does add 1 megabyte to your apps).
So in fact, you can now create and populate a full SQLite DB and just ship it to your server “as is”. Add encryption and you have a pretty good syncronization layer to play with.
W3C, WC3 or just WC
SQLite used to be built into modern browsers, then dubbed “WebSQL”; but in their infinite wisdom the W3C decided to deprecate that and go for a key-value pair storage system reminicient of MongoDB on wellfare. The good news is that our RTL is now so powerful that we dont need to care about the whims of the W3C any more. Just like Delphi revolutionized database driven desktop application development in the 90’s — we will revolutionize DB driven JavaScript applications for the modern age.
True binary streams? You got it! Compression routines ported directly from Delphi? No problem. In-memory databases? Yup!
Websockets made human
Websocket, oh what a magical buzzword! With all the hype you should think it was black magic. Well our SmartCL.Inet unit has now been expanded with a very cool WebSocket client. This means that if you want to talk directly with a nodeJS installation, or perhaps even better — write your own websocket server and get into Amazon cloud services for next to nothing, the RTL now makes that possible.
Writing a websocket client is now reduced to:
FSocket.Connect("wss://echo.websocket.org",[]);
FSocket.OnOpen:=Procedure (Sender:TW3WebSocket)
begin
writeln("Socket connected");
Sender.Write("Rock it with HTML5 WebSocket");
end;
FSocket.OnClosed:=procedure (Sender:TW3WebSocket)
begin
Writeln("Socket disconnected");
end;
FSocket.OnMessage:=Procedure (Sender:TW3WebSocket;Data:String)
begin
Writeln("Data received:" + Data);
end;
FSocket.OnError:=Procedure (Sender:TW3WebSocket)
begin
writeln("Socket has errors");
end;
File storage for the rest of us
Sandboxed files and JavaScript is a complete mess. JavaScript is utterly event driven, wich means that everything requires callback handlers. From accessing a medium, to being allowed said access to creating a writer and finally do the actual writing (phew!). And each event handler must denote the calling context just for the hell of it.
To make this usable by human beings, the RTL introduces a concept called “file-actions”. They work more or less like TAction under Delphi, but are especially tailored for loading and saving data in a uniform way.
Here is an example of saving and then loading back a small file:
procedure TForm1.W3Button4Click(Sender: TObject);
var
mAction: TFileActivitySave;
begin
mAction:=TFileActivitySave.Create;
mAction.FileName:="document.txt";
mAction.FileMode:=fmPermanent;
mAction.OnExecute:=procedure (sender:Tobject)
begin
writeln("Saving done");
var mLoader:=TFileActivityLoad.Create;
mLoader.FileData:=TMemoryStream.Create;
mLoader.FileName:='document.txt';
mLoader.FileMode:=fmPermanent;
mLoader.OnExecute:=Procedure (sender:TObject)
begin
writeln("Loading done!");
writeln("Bytes ready = " + mLoader.FileData.Size.tostring);
try
var mReader:=TReader.Create(mLoader.FileData);
writeln(mReader.ReadString);
except
on e: exception do
writeln(e.message);
end;
end;
mLoader.Execute;
end;
mAction.FileData:=TMemoryStream.Create;
var mTemp:=TWriter.Create(mAction.FileData);
mTemp.WriteString("This is some data");
mAction.Execute;
end;
As you begin to explore the additions to the RTL in the next release, you will find that a lot of time and effort has gone into this. So I hope everyone get’s excited when they what they can now do — and that object pascal is the perfect language for HTML5 and mobile application development!
Hire a Delphi superhero!
This is your chance to hire a Delphi super-hero, namely me! That’s right, I’m presently in the market for a new full-time job!
Note: if you need people for a short project then I personally must decline, but I can forward your position at Delphi Developer (4.000 Delphi developers) and here on my website (average of 15.000 visitors per month)
I must underline the obvious, namely that I live in Norway with no intention of moving. However, I have a fully established office which has been used for three consecutive years. So I am used to working remotely, communicating through Skype, Messenger, Google Meetup’s and similar utilities. This has worked brilliantly so far. Full time employment over the internet is finally a reality.
For Norwegian companies I can offer 50% commuting (100% in Vestfold) from Oslo to Akershus in the north, to Sandefjord and Larvik in the south. The reason I can only offer 50% is because I have custody for two children every other week, which means I am unable to drive long distances in combination with school and similar factors.
Jon Lennart Aasenden, 41 years old senior software engineer using Delphi
My experience is too long to list here, but if you are in the market for a dedicated software engineer then you have come to the right place. Simply drop me a line on e-mail (address at the end of this post) and I will get back to you with a full resume ASAP.
In short: I have worked with Delphi for 15 years, I am the author of Smart Mobile Studio and wrote the VJL (run-time library, similar to VCL) from scratch. I have worked for some of the biggest companies in Norway. I have good papers, excellent reputation and is recognized for my skill, positive attitude and “yes we can” mentality.
Before my 15 years of Delphi I spent roughly the same amount of time programming games, multimedia and contributing to the demo scene. In my view, demo programming is one of the most rewarding intellectual exercises you can engage in. It teaches you how to think and prepares you for “real life solutions” as a professional.
In my career as a Delphi developer I have written several commercial products; ranging from a complete invoice and credit management application, games and multimedia products, serial management components for Delphi, backup clients and much, much more.
No matter what your company works with, as long as it’s Delphi I’m the right man for the job.
My kind of job
It goes without saying that I am looking for a purely Delphi centric position. Although I have worked with C++ and C# and have no problems navigating and adding to such a codebase — there can be no doubt that Delphi is the environment where I excel at my work.
I bring with me a solid insight into HTML5/JavaScript and am willing to teach and educate your existing staff in Smart Mobile Studio, which I am also the founder and inventor of; bringing your team up to speed with the next step in computing: namely distributed cloud computing. At which JavaScript and Object Pascal play central roles.
Torro Invoice
So if you are moving towards cloud computing or want to leverage the cloud for service oriented architectures using RPC or REST servers, I am the guy you want to talk to. Especially if you want to use object pascal and leverage nodeJS, JavaScript and the whole spectrum of mobile devices. In short, with me on your team you will have a huge advantage.
What you get is 15 years of Delphi experience; and before that, 15 years of software authoring, demo and game development on 16 bit machines.
Good qualities
I have included a small list of “ad-hoc” qualifications. The list below does not serve as anything other than indicators. I have been a professional programmer for many, many years and to list every single piece of technology is pointless. But hopefully the list is enough to give you an idea of my qualifications, depth and diversity.
- Write custom Delphi controls and UI widgets
- Create control packages and resource management for our projects
- Optimize procedures by hand by refactoring variables, parameters and datatypes.
- Write a database engine from scratch and also use existing engines such as Elevatedb, MSSQL, Oracle, Firebird, SQLite and MySQL
- Work with Windows API directly
- Work with freepascal and port code to Linux and Macintosh
- Write networking libraries, custom protocols and high-speed multithreaded servers
- Use classical frameworks like Indy, Internet Direct and other out of box solutions
- Write advanced parsers for source-code and any complex text-format
- Write a real-life compiler and IDE (www.smartmobilestudio.com and quartexpascal.wordpress.com)
- Create a graphics library from scratch, supporting different pixel formats
- Work with large files, performing complex operations on gigabytes and terabytes
- Write mobile and desktop applications using Smart Mobile Studio and FireMonkey
- Create your server environment, architect your RPC (remote procedure call) service infrastructure
- Implement custom services through nodeJS, Smart Mobile Studio, Delphi and FreePascal
- Use Remobjects product line: such as RemObjects SDK, Hydra and Data-abstract
- Link with C and C++ object files
- Use project management tools, scrum, SVN, github and package management
- Write, export and import DLL files; both written in and imported to Delphi
- Write solid architectures using moderns technologies: xsl schemas, stylesheet validation, rest, node services
.. and much, much more!
Note: While you may not have a need for all of these qualifications (or any one of them), the understanding which the listed topics demand that I master is of great value to you as an employer. The more knowledge and hands-on experience a programmer can offer, the better qualified he is to deal with real-life problems.
Tools of the trade
All programmers have a toolbox with frameworks, code collections or third party components they use. These are the products I always try to use in projects. I have a huge library of components I have bought over the years, and I have no problem getting to know whatever toolkit you prefer. Below is a short list of my favorite tools, things I always carry with me to work.
- RemObjects SDK to isolate business logic in RPC servers, this allows us to use the same back-end for both desktop, mobile and html5 javascript.
- RemObject Data Abstract
- Remobjects Hydra
- ElevateDB and DBIsam (DBIsam for Delphi 7-2010, ElevateDB for the rest)
- Developer Express component suite
- TMS Aurelius database framework
- MyDac – MySQL data access components
- MSDac – Microsoft SQL server data access components
- Raize component suite
- FreePascal and Lazarus for OS X and Linux services
- Mono/C# when customers demands it (or Visual Studio)
- Firemonkey when that is in demand
- Smart Mobile Studio for mobile applications, cloud services [nodejs] and HTML5/Web
Tools and utilities
- Beyond compare (compare versions of the same file)
- Total commander (excellent search inside files in a directory, recursive)
- DBGen – A program written by me for turning Interbase/firebird databases into classes. Essentially my twist on Aurelius from TMS, except here you generate fixed classes in XML Data Mapping style.
Note: For large projects which needs customization I tend to write tools and converters myself. The DBgen tool was created when I worked at a company which used Firebird. They had two people who hand wrote wrapper classes for nearly 100 tables (!) They had used almost two months and covered about half. I wrote the generator in two days and it generates perfect wrappers for all tables in less than 10 minutes.
Safety first
As a father of three I am only interested in stable, long-term employments. Quick projects (unless we are talking six figures) are of no interest, nor do I want to work for startup groups or kickstarters. I am quite frankly not in a position to take risks; children comes first.
Many of my products are available for purchase online
I am used to working from my own office. I was initially skeptical to that, but it has turned out to work brilliantly. I am always available (within working hours Norwegian time) on Skype, Google, Messenger and phone.
Languages work format
For the past year I have been working full time with C# under Microsoft Visual Studio. Before that I used Mono for about four months, which I must admit I prefer over Visual Studio, and wrote a few iPhone and iPad apps. So I am not a complete stranger to C# or other popular languages. But as I have underlined in my posts on countless occations – object pascal is a better language. It is truly sad that Borland was brought down all those years ago, because with them they dragged what must be one of the most profound and influential languages of our time.
That said, I have no scruples about mixing and matching languages. Be it Remobjects Oxygene, C#, C++ or Object Pascal. I also do some mocking in BlitzBasic which is without question the fastest (execution wise) of the bunch, but not really suitable for enterprise level applications.
The point here being that I am versatile, prepared to learn new things and not afraid to ask. It would be a miracle if a man could walk through life without learning something new, so it’s important to show respect and be remain humble. Different languages represents different approaches to problem solving, and that is an important aspect of what it means to be a programmer.
The only thing I ask is that I am properly informed regarding the language you expect me to deliver in – and that I receive the required training or courseware in due time. Do not expect super-hero status in C#, I am absolutely above average, but it cannot be compared to my efficiency and excellence using Delphi.
Contact
If you are interested in hiring me for your company, then don’t hesitate to contact me for a resume. You may e-mail me at lennart.aasenden AT gmail dot com.
Jon L. Aasenden 
You must be logged in to post a comment.