Archive

Archive for the ‘CSS’ Category

Amibian.js under the hood

December 5, 2018 2 comments

Amibian.js is gaining momentum as more and more developers, embedded systems architects, gamers and retro computer enthusiasts discover the project. And I have to admit I’m pretty stoked about what we are building here myself!

intro

In a life-preserver no less ūüėÄ

But, with any new technology or invention there are two common traps that people can fall into: The first trap is to gravely underestimate a technology. JavaScript certainly invites this, because only a decade ago the language was little more than a toy. Since then JavaScript have evolved to become the most widely adopted programming language in the world, and runtime engines like Google’s V8 runs JavaScript almost as fast as compiled binary code (“native” means machine code, like that produced by a C/C++ compiler, Pascal compiler or anything else that produces programs that run under Linux or Windows).

It takes some adjustments, especially for traditional programmers that havent paid attention to where browsers have gone – but long gone are the days of interpreted JavaScript. Modern JavaScript is first parsed, tokenized and compiled to bytecodes. These bytecodes are then JIT compiled (“just in time”, which means the compilation takes place inside the browser) to real machine-code using state of the art techniques (LLVM). So the JavaScript of 2018 is by no means the JavaScript of 2008.

The second trap you can fall into – is to exaggerate what a new technology can do, and attach abilities and expectations to a product that simply cannot be delivered.¬†It is very important to me that people don’t fall into either trap, and that everyone is informed about what Amibian.js actually is and can deliver – but also what it wont deliver. Rome was not built-in a day, and it’s wise to study all the factors before passing judgement.

I have been truly fortunate that people support the project financially via Patreon, and as such I feel it’s my duty to document and explain as much as possible. I am a programmer and I often forget that not everyone understands what I’m talking about. We are all human and make mistakes.

Hopefully this post will paint a clearer picture of Amibian.js and what we are building here. The project is divided into two phases: first to finish Amibian.js itself, and secondly to write a Visual Studio clone that runs purely in the browser. Since it’s easy to mix these things up, I’m underlining this easy – just in case.

What the heck is Amibian.js?

Amibian.js is a group of services and libraries that combined creates a portable operating-system that renders to HTML5. A system that was written using readily available web technology, and designed to deliver advanced desktop functionality to web applications.

The services that make up Amibian.js was designed to piggyback on a thin Linux crust, where Linux deals with the hardware, drivers and the nitty-gritty we take for granted. There is no point trying to write a better kernel in 2018, because you are never going to catch up with Linus Torvalds. It’s must more interesting to push modern web technology to the absolute limits, and build a system that is truly portable and distributed.

smart_ass

Above: Amibian.js is created in Smart Pascal and compiled to JavaScript

The service layer is written purely in node.js (JavaScript) which guarantees the same behavior regardless of host platform. One of the benefits of using off-the-shelves web technology is that you can physically copy the whole system from one machine to the other without any changes. So if you have a running Amibian.js system on your x86 PC, and copy all the files to an ARM computer – you dont even have to recompile the system. Just fire up the services and you are back in the game.

Now before you dismiss this as “yet another web mockup” please remember what I said about JavaScript: the JavaScript in 2018 is not the JavaScript of 2008. No other language on the planet has seen as much development as JavaScript, and it has evolved from a “browser toy” – into the most important programming language of our time.

So Amibian.js is not some skin-deep mockup of a desktop (lord knows there are plenty of those online). It implements advanced technologies such as remote filesystem mapping, an object-oriented message protocol (Ragnarok), RPCS (remote procedure call invocation stack), video codec capabilities and much more — all of it done with JavaScript.

In fact, one of the demos that Amibian.js ships with is Quake III recompiled to JavaScript. It delivers 120 fps flawlessly (browser is limited to 60 fps) and makes full use of standard browser technologies (WebGL).

utube

Click on picture above to watch Amibian.js in action on YouTube

So indeed, the JavaScript we are talking about here is cutting edge. Most of Amibian.js is compiled as “Asm.js” which means that the V8 runtime (the code that runs JavaScript inside the browser, or as a program under node.js) will JIT compile it to highly efficient machine-code.

Which is why Amibian.js is able to do things that people imagine impossible!

Ok, but what does Amibian.js consist of?

Amibian.js consists of many parts, but we can divide it into two categories:

  • A HTML5 desktop client
  • A system server and various child processes

These two categories have the exact same relationship as the X desktop and the Linux kernel. The client connects to the server, invokes procedures to do some work, and then visually represent the response This is identical to how the X desktop calls functions in the kernel or one of the Linux libraries. The difference between the traditional, machine code based OS and our web variation, is that our version doesn’t have to care about the hardware. We can also assign many different roles to Ambian.js (more about that later).

smartdesk

Enjoying other cloud applications is easy with Amibian.js, here is Plex, a system very much based on the same ideas as Amibian.js

And for the record: I’m¬†trying to avoid a bare-metal OS, otherwise I would have written the system using a native programming language like C or Object-Pascal. So I am not using JavaScript because I lack skill in native languages, I am using JavaScript because native code is not relevant for the tasks Amibian.js solves. If I used a native back-end I could have finished this in a couple of months, but a native server would be unable to replicate itself between cloud instances because chipset and CPU would be determining factors.

The Amibian.js server is not a single program. The back-end for Amibian.js consists of several service applications (daemons on Linux) that each deliver specific features. The combined functionality of these services make up “the amibian kernel” in our analogy with Linux. You can think of these services as the library files in a traditional system, and programs that are written for Amibian.js can call on these to a wide range of tasks. It can be as simple as reading a file, or as complex as registering a new user or requesting admin rights.

The greatest strength of Amibian.js is that it’s designed to run clustered, using as many CPU cores as possible. It’s also designed to scale, meaning that it will replicate itself and divide the work between different instances. This is where things get’s interesting, because an Amibian.js cluster doesn’t need the latest and coolest hardware to deliver good performance. You can build a cluster of old PC’s in your office, or a handful of embedded boards (ODroid XU4, Raspberry PI’s and Tinkerboard are brilliant candidates).

But why Amibian.js? Why not just stick with Linux?

That is a fair question, and this is where the roles I mentioned above comes in.

As a software developer many of my customers work with embedded devices and kiosk systems. You have companies that produce routers and set-top boxes, NAS boxes of various complexity, ticket systems for trains and busses; and all of them end up having to solve the same needs.

What each of these manufacturers have in common, is the need for a web desktop system that can be adapted for a specific program. Any idiot can write a web application, but when you need safe access to the filesystem, unified API’s that can delegate signals to Amazon, Azure or your company server, things suddenly get’s more complicated. And even when you have all of that, you still need a rock solid application model suitable for distributed computing. You might have 1 ticket booth, or 10.000 nation wide. There are no systems available that is designed to deal with web-technology on that scale. Yet ūüėČ

Let’s look at a couple of real-life scenarios that I have encountered, I’m confident you will recognize a common need. So here are some roles that Amibian.js can assume and help deliver a solution rapidly. It also gives you some ideas of the economic possibilities.

Updated: Please note that we are talking javascript here, not native code. There are a lot of native solutions out there, but the whole point here is to forget about CPU, chipset and target and have a system floating on top of whatever is beneath.

  • When you want to change some settings on your router – you login to your router. It contains a small apache server (or something similar) and you do all your maintenance via that web interface. This web interface is typically skin-deep, annoying to work with and a pain for developers to update since it’s connected to a native apache module which is 100% dependent on the firmware. Each vendor end up re-inventing the wheel over and over again.
  • When you visit a large museum notice the displays. A museum needs to display multimedia, preferably on touch capable devices, throughout the different exhibits. The cost of having a developer create native applications that displays the media, plays the movies and gives visual feedback is astronomical. Which is why most museums adopt web technology to handle media presentation and interaction. Again they re-invent the wheel with varying degree of success.
  • Hotels have more or less the exact same need but on a smaller scale, especially the larger hotels where the lobby have information booths, and each room displays a web interface via the TV.
  • Shopping malls face the same challenge, and depending on the size they can need anything from a single to a hundred nodes.
  • Schools and education spend millions on training software and programming languages every year. Amibian.js can deliver both and the schools would only pay for maintenance and adaptation – the product itself is free. Kids get the benefit of learning traditional languages and enjoying instant visual feedback! They can learn Basic, Pascal, JavaScript and C. I firmly believe that the classical languages will help make them better programmers as they evolve.

You are probably starting to see the common denominator here?

They all need a web-based desktop system, one that can run complex HTML5 based media applications and give them the same depth as a native operating-system; Which is pretty hard to achieve with JavaScript alone.

Amibian.js provides a rich foundation of more than 4000 classes that developers can use to write large, complex and media rich applications (see Smart Mobile Studio below). Just like Linux and Windows provides a wealth of libraries and features for native application development – Amibian.js aims to provide the same for cloud and embedded systems.

And as the name implies, it has roots in the past with the machine that defined multimedia, namely the Commodore Amiga. So the relation is more than just visually, Amibian.js uses the same system architecture – because we believe it’s one of the best systems ever designed.

If JavaScript is so poor, why should we trust you to deliver so much?

First of all I’m not selling anything. It’s not like this project is something that is going to make me a ton of cash. I ask for support during the development period because I want to allocate proper time for it, but when done Amibian.js will be free for everyone (LGPL). And I’m also writing it because it’s something that I need and that I havent seen anywhere else. I think you have to write software for yourself, otherwise the quality wont be there.

Secondly, writing Amibian.js in raw JavaScript with the same amount of functions and depth would take years. The reason I am able to deliver so much functionality quickly, is because I use a compiler system called Smart Mobile Studio. This saves months and years of development time, and I can use all the benefits of OOP.

Prior to starting the Amibian.js project, I spent roughly 9 years creating Smart Mobile Studio. Smart is not a solo project, many individuals have been involved – and the product provides a compiler, IDE (editor and tools), and a vast run-time library of pre-made classes (roughly 4000 ready to use classes, or building-blocks).

amibian_shell

Writing large-scale node.js services in Smart is easy, fun and powerful!

Unlike other development systems, Smart Mobile Studio compiles to JavaScript rather than machine-code. We have spent a great deal of time making sure we could use proper OOP (object-oriented programming), and we have spent more than 3 years perfecting a visual application framework with the same depth as the VCL or FMX (the core visual frameworks for C++ builder and Delphi).

The result is that I can knock out a large application that a normal JavaScript coder would spend weeks on – in a single day.

Smart Mobile Studio uses the object-pascal language, a dialect which is roughly 70% compatible with Delphi. Delphi is exceptionally well suited for writing large, data driven applications. It also thrives for embedded systems and low-level system services. In short: it’s a lot easier to maintain 50.000 lines of object pascal code, than 500.000 lines of JavaScript code.

Amibian.js, both the service layer and the visual HTML5 client application, is written completely using Smart Mobile Studio. This gives me as the core developer of both systems a huge advantage (who knows it better than the designer right?). I also get to write code that is truly OOP (classes, inheritance, interfaces, virtual and abstract methods, partial classes etc), because our compiler crafts something called a VMT (virtual method table) in JavaScript.

Traditional JavaScript doesn’t have OOP, it has something called prototypes. With Smart Pascal I get to bring in code from the object-pascal community, components and libraries written in Delphi or Freepascal – which range in the hundreds of thousands. Delphi alone has a massive library of code to pick from, it’s been a popular toolkit for ages (C is 3 years older than pascal).

But how would I use Amibian.js? Do I install it or what?

Amibian.js can be setup and used in 4 different ways:

  • As a true desktop, booting straight into Amibian.js in full-screen
  • As a cloud service, accessing it through any modern browser
  • As a NAS or Kiosk front-end
  • As a local system on your existing OS, a batch script will fire it up and you can use your browser to access it on https://127.0.0.1:8090

So the short answer is yes, you install it. But it’s the same as installing Chrome OS. It’s not like an application you just install on your Linux, Windows or OSX box. The whole point of Amibian.js is to have a platform independent, chipset agnostic system. Something that doesn’t care if you using ARM, x86, PPC or Mips as your CPU of preference. Developers will no doubt install it on their existing machines, Amibian.js is non-intrusive and does not affect or touch files outside its own eco-system.

But the average non-programmer will most likely setup a dedicated machine (or several) or just deploy it on their home NAS.

The first way of enjoying Amibian.js is to install it on a PC or ARM device. A disk image will be provided for supporters so they can get up and running ASAP. This disk image will be based on a thin Linux setup, just enough to get all the drivers going (but no X desktop!). It will start all the node.js services and finally enter a full-screen web display (based on Chromium Embedded) that renders the desktop. This is the method most users will prefer to work with Amibian.js.

The second way is to use it as a cloud service. You install Amibian.js like mentioned above, but you do so on Amazon or Azure. That way you can login to your desktop using nothing but a web browser. This is a very cost-effective way of enjoying Amibian.js since renting a virtual instance is affordable and storage is abundant.

The third option is for developers. Amibian.js is a desktop system, which means it’s designed to host more elaborate applications. Where you would normally just embed an external website into an IFrame, but Amibian.js is not that primitive. Hosting external applications requires you to write a security manifest file, but more importantly: the application must interface with the desktop through the window’s message-port. This is a special object that is sent to the application as a hand-shake, and the only way for the application to access things like the file-system and server-side functionality, is via this message-port.

Calling “kernel” level functions from a hosted application is done purely via the message-port mentioned above. The actual message data is JSON and must conform to the Ragnarok client protocol specification. This is not as difficult as it might sound, but Amibian.js takes security very seriously – so applications trying to cause damage will be promptly shut down.

You mention hosted applications, do you mean websites?

Both yes and no: Amibian.js supports 3 types of applications:

  • Ordinary HTML5/JS based applications, or “websites” as many would call them. But like I talked about above they have to establish a dialog with the desktop before they can do anything useful.
  • Hybrid applications where half is installed as a node.js service, and the other half is served as a normal HTML5 app. This is the coolest program model, and developers essentially write both a server and a client – and then deploy it as a single package.
  • LDEF compiled bytecode applications, a 68k inspired assembly language that is JIT compiled by the browser (commonly called “asm.js”) and runs extremely fast. The LDEF virtual machine is a sub-project in Amibian.js

The latter option, bytecodes, is a bit like Java. A part of the Amibian.js project is a compiler and runtime system called LDEF.

patron_asm2

Above: The Amibian.js LDEF assembler, here listing opcodes + disassembling a method

The first part of the Amibian.js project is to establish the desktop and back-end services. The second part of the project is to create the worlds first cloud based development platform. A full Visual Studio clone if you like, that allows anyone to write cloud, mobile and native applications directly via the browser (!)

Several languages are supported by LDEF, and you can write programs in Object Pascal, Basic and C. The Basic dialect is especially fun to work with, since it’s a re-implementation of BlitzBasic (with a lot of added extras). Amiga developers will no doubt remember BlitzBasic, it was used to create some great games back in the 80s and 90s. It’s well suited for games and multimedia programming and above all – very easy to learn.

More advanced developers can enjoy Object Pascal (read: Delphi) or a sub-set of C/C++.

And please note: This IDE is designed for large-scale applications, not simple snippets. The ultimate goal of Amibian.js is to move the entire development cycle to the cloud and away from the desktop. With Amibian.js you can write a cool “app” in BlitzBasic, run it right in the browser — or compile it server-side and deploy it to your Android Phone as a real, natively compiled application.

So any notion of a “mock desktop for HTML” should be firmly put to the side. I am not playing around with this product and the stakes are very real.

But why don’t you just use ChromeOS?

There are many reasons, but the most important one is chipset independence. Chrome OS is a native system, meaning that it’s core services are written in C/C++ and compiled to machine code. The fundamental principle of Amibian.js is to be 100% platform agnostic, and “no native code allowed”. This is why the entire back-end and service layer is targeting node.js. This ensures the same behavior regardless of processor or host system (Linux being the default host).

Node.js has the benefit of being 100% platform independent. You will find node.js for ARM, x86, Mips and PPC. This means you can take advantage of whatever hardware is available. You can even recycle older computers that have lost mainstream support, and use them to run Amibian.js.

A second reason is: Chrome OS might be free, but it’s only as open as Google want it to be. ChromeOS is not just something you pick up and start altering. It’s dependence on native programming languages, compiler toolchains and a huge set of libraries makes it extremely niche. It also shields you utterly from the interesting parts, namely the back-end services. It’s quite frankly boring and too boxed in for any practical use; except for Google and it’s technology partners that is.

I wanted a system that I could move around, that could run in the cloud, on cheap SBC’s. A system that could scale from handling 10 users to 1000 users – a system that supports clustering and can be installed on multiple machines in a swarm.

A system that anyone with JavaScript knowledge can use to create new and exciting systems, that can be easily expanded and serve as a foundation for rich media applications.

What is this Amiga stuff, isn’t that an ancient machine?

In computing terms yes, but so is Unix. Old doesn’t automatically mean bad, it actually means that it’s adapted and survived challenges beyond its initial design. While most of us remember the Amiga for its games, I remember it mainly for its elegant and powerful operating-system. A system so flexible that it’s still in use around the world – 33 years after the machine hit the market. That is quite an achievement.

image2

The original Amiga OS, not bad for a 33-year-old OS! It was and continues to be way ahead of everyone else. A testament to the creativity of its authors

Amibian.js as the name implies, borrows architectural elements en-mass from Amiga OS. Quite simply because the way Amiga OS is organized and the way you approach computing on the Amiga is brilliant. Amiga OS is much more intuitive and easier to understand than Linux and Windows. It’s a system that you could learn how to use fully with just a couple of days exploring; and no manuals.

But the similarities are not just visual or architectural. Remember I wrote that hosted applications can access and use the Amibian.js services? These services implement as much of the original ROM Kernel functions as possible. Naturally I can’t port all of it, because it’s not really relevant for Amibian.js. Things like device-drivers serve little purpose for Amibian.js, because Amibian.js talks to node.js, and node talks to the actual system, which in turn handles hardware devices. But the way you would create windows, visual controls, bind events and create a modern, event-driven application has been preserved to the best of my ability.

But how does this thing boot? I thought you said server?

If you have setup a dedicated machine with Amibian.js then the boot sequence is the same as Linux, except that the node.js services are executed as background processes (daemons or services as they are called), the core server is initialized, and then a full-screen HTML5 view is set up that shows the desktop.

But that is just for starting the system. Your personal boot sequence which deals with your account, your preferences and adaptations – that boots when you login to the system.

When you login to your Amibian.js account, no matter if it’s just locally on a single PC, a distributed cluster, or via the browser into your cloud account — several things happen:

  1. The client (web-page if you like) connects to the server using WebSocket
  2. Login is validated by the server
  3. The client starts loading preferences files via the mapped filesystem, and then applies these to the desktop.
  4. A startup-sequence script file is loaded from your account, and then executed. The shell-script runtime engine is built into the client, as is REXX execution.
  5. The startup-script will setup configurations, create symbolic links (assigns), mount external devices (dropbox, google drive, ftp locations and so on)
  6. When finished the programs in the ~/WbStartup folder are started. These can be both visual and non-visual.

As you can see Amibian.js is not a mockup or “fake” desktop. It implements all the advanced features you expect from a “real” desktop. The filesystem mapping is especially advanced, where file-data is loaded via special drivers; drivers that act as a bridge between a storage service (a harddisk, a network share, a FTP host, Dropbox or whatever) and the desktop. Developers can add as many of these drivers as they want. If they have their own homebrew storage system on their existing servers, they can implement a driver for it. This ensures that Amibian.js can access any storage device, as long as the driver conforms to the driver standard.

In short, you can create, delete, move and copy files between these devices just like you do on Windows, OSX or the Linux desktop. And hosted applications that run inside their own window can likewise request access to these drivers and work with the filesystem (and much more!).

Wow this is bigger than I thought, but what is this emulation I hear about? Can Amibian.js really run actual programs?

Amibian.js has a JavaScript port of UAE (Unix Amiga Emulator). This is a fork of SAE (scripted Amiga Emulator) that has been heavily optimized for web. Not only is it written in JavaScript, it performs brilliantly and thus allows us to boot into a real Amiga system. So if you have some floppy-images with a game you love, that will run just fine in the browser. I even booted a 2 gigabyte harddisk image ūüôā

But Amiga emulation is just the beginning. More and more emulators are ported to JavaScript; you have Nes, SNes, N64, PSX I & II, Sega Megadrive and even a NEO GEO port. So playing your favorite console games right in the browser is pretty straight forward!

But the really interesting part is probably QEmu. This allows you to run x86 instances directly in the browser too. You can boot up in Windows 7 or Ubuntu inside an Amibian.js window if you like. Perhaps not practical (at this point) but it shows some of the potential of the system.

I have been experimenting with a distributed emulation system, where the emulation is executed server-side, and only the graphics and sound is streamed back to the Amibian.js client in real-time. This has been possible for years via Apache Guacamole, but doing it in raw JS is more fitting with our philosophy: no native code!

I heard something about clustering, what the heck is that?

Remember I wrote about the services that Amibian.js has? Those that act almost like libraries on a physical computer? Well, these services don’t have to be on the same machine — you can place them on separate machines and thus its able to work faster.

47470965_10155861938320906_4959664457727868928_n

Above: The official Amibian.js cluster, 4 x ODroid XU4s SBC’s in a micro-rack

A cluster is typically several computers connected together, with the sole purpose of having more CPU cores to divide the work on. The cool thing about Amibian.js is that it doesn’t care about the underlying CPU. As long as node.js is available it will happily run whatever service you like – with the same behavior and result.

The official Amibian.js cluster consists of 5 ODroid XU4/S SBC (single board computers). Four of these are so-called “headless” computers, meaning that they don’t have a HDMI port – and they are designed to be logged into and software setup via SSH or similar tools. The last machine is a ODroid XU4 with a HDMI out port, which serves as “the master”.

The architecture is quite simple: We allocate one whole SBC for a single service, and allow the service to copy itself to use all the CPU cores available (each SBC has 8 CPU cores). With this architecture the machine that deals with the desktop clients don’t have to do all the grunt work. It will accept tasks from the user and hosted applications, and then delegate the tasks between the 4 other machines.

Note: The number of SBC’s is not fixed. Depending on your use you might not need more than a single SBC in your home setup, or perhaps two. I have started with 5 because I want each part of the architecture to have as much CPU power as possible. So the first “official” Amibian.js setup is a 40 core monster shipping at around $250.

But like mentioned, you don’t have to buy this to use Amibian.js. You can install it on a single spare X86 PC you have, or daisy chain a couple of older PC’s on a switch for the same result.

Why Headless? Don’t you need a GPU?

The headless SBC’s in the initial design all have GPU (graphical processing unit) as well as audio capabilities. What they lack is GPIO pins and 3 additional USB ports. So each of the nodes on our cluster can handle graphics at blistering speed — but that is ultimately not their task. They serve more as compute modules that will be given tasks to finish quickly, while the main machine deals with users, sessions, traffic and security.

The 40 core cluster I use has more computing power than northern europe had in the early 80s, that’s something to think about. And the pricetag is under $300 (!). I dont know about you but I always wanted a proper mainframe, a distributed computing platform that you can login to and that can perform large tasks while I do something else. This is as close as I can get on a limited budget, yet I find the limitations thrilling and fun!

Part of the reason I have opted for a clustered design has to do with future development. While UAE.js is brilliant to emulate an Amiga directly in the browser – a more interesting design is to decouple the emulation from the output. In other words, run the emulation at full speed server-side, and just stream the display and sounds back to the Amibian.js display. This would ensure that emulation, of any platform, runs as fast as possible, makes use of multi-processing (read: multi threading) and fully utilize the network bandwidth within the design (the cluster runs on its own switch, separate from the outside world-wide-web).

I am also very interested in distributed computing, where we split up a program and run each part on different cores. This is a topic I want to investigate further when Amibian.js is completed. It would no doubt require a re-design of the LDEF bytecode system, but this something to research later.

Will Amibian.js replace my Windows box?

That depends completely on what you use Windows for. The goal is to create a self-sustaining system. For retro computing, emulation and writing cool applications Amibian.js will be awesome. But Rome was not built-in a day, so it’s wise to be patient and approach Amibian.js like you would Chrome OS. Some tasks are better suited for native systems like Linux, but more and more tasks will run just fine on a cloud desktop like Amibian.js.

Until the IDE and compilers are in place after phase two, the system will be more like an embedded OS. But when the LDEF compiler and IDE is in place, then people will start using it en-mass and produce applications for it. It’s always a bit of work to reach that point and create critical mass.

tomes

Object Pascal is awesome, but modern, native development systems are quite demanding

My personal need has to do with development. Some of the languages I use installs gigabytes onto my PC and you need a full laptop to access them. I love Amibian.js because I will be able to work anywhere in the world, as long as a browser and normal internet line is available. In my case I can install a native compiler on one of the nodes in the cluster, and have LDEF emit compatible code; voila, you can build app-store ready applications from within a browser environment.

 

I also love that I can set-up a dedicated platform that runs legacy applications, games – and that I can write new applications and services using modern, off the shelve languages. And should a node in the cluster break down, I can just copy the whole system over to a new, affordable SBC and keep going. No super expensive hardware to order, no absurd hosting fees, and finally a system that we all can shape and use in a plethora of systems. From a fully fledged desktop to a super advanced NAS or Router that use Amibian.js to give it’s customers a fantastic experience.

And yes, I get to re-create the wonderful reality of Amiga OS without the absurd egoism that dominates the Amiga owners to this day. I don’t even know where to begin with the present license holders – and I am so sick of the drama that rolling my own seemed the only reasonable path forward.

Well — I hope this helps clear up any misconceptions about Amibian.js, and that you find this as interesting as I do. As more and more services are pushed cloud-side, the more relevant Amibian.js will become. It is perfect as a foundation for large-scale applications, embedded systems — and indeed, as a solo platform running on embedded devices!

I cant wait to finish the services and cluster this sucker on the ODroid rack!

If you find this project interesting, head over to my Patreon website and get involved! I could really use your support, even if it’s just a $5 “high five”. Visit the project at:¬†http://www.patreon.com/quartexNow

Smart Mobile Studio: Q&A about v3.0 and beyond

July 1, 2018 4 comments

A couple of days back I posted a sneak-peek of our upcoming Smart Mobile Studio 3.0 web desktop framework; as a consequence my Facebook messenger app has practically exploded with questions.

smart_desktop

The desktop client / server framework is an example of what you can do in Smart

As you can imagine, the questions people ask are often very similar; so similar in fact that I will answer the hottest topics here. Hopefully that will make it easier for everyone.

If you have further questions then either ask them on our product forums or the Delphi Developer group on Facebook.

 

Generics

Yes indeed we have generics running in the labs. We havent set a date on when we will merge the new compiler-core, but it’s not going to happen until (at the earliest) v3.4. So it’s very much a part of Smart’s future but we have a couple of steps left on our time-line for v3.0 through v3.4.

RTTI access

RTTI is actually in the current version, but sadly there is a bug there that causes the code generator to throw a fit. The fix for this depends on a lot of the sub-strata in the new compiler-core, so it will be available when generics is available.

Associative arrays

This is ready and waiting in the new core, so it will appear together with generics and RTTI.

Databases

We have supported databases since day 1, but the challenge with JavaScript is that there are no “standards” like we are used to from established systems like Delphi or Lazarus.

Under the browser we support WebSQL and our own TW3Dataset. We also compiled SQLite from native C to JavaScript so we can provide a fast, lightweight SQL engine for the browser regardless of what the W3C might do (WebSQL has been deprecated but will be around for many years still).

Server side it’s a whole different ballgame. There you have drivers (or modules) for every possible database you can think of, even Firebird. But each module is implemented as the authors see fit. This is where our Database framework comes in, sets a standard, and we then inherit out classes and implement the engines we want.

This framework and standard is being written now, but it wont be introduced until v3.1 and v3.2. In the meantime you have sqlite both server-side and client-side, WebSQL and TW3Dataset.

Attributes

This question is often asked separately from RTTI, but it’s ultimately an essential part of what RTTI delivers.

So same answer: it will arrive with the new compiler-core / infrastructure.

Server-side scripting

22555292_1630289757034374_6911478701417326545_n

The new theme system in action

While we do see how this could be useful, it requires a substantial body of work to make a reality. Not only would we have to implement the whole “system” namespace from scratch since JavaScript would not be present, but we would also have to introduce a a secondary namespace; one that would be incompatible with the whole RTL at a fundamental level. Instead of going down this route we opted for Node.js where creating the server itself is the norm.

 

If we ever add server-side scripting it would be JavaScript support under node.js by compiling the V8 engine from C to asm.js. But right now our focus is not on server-side-scripting, but on cloud building-blocks.

Bytecode compilation

I implemented the assembler and runtime for our bytecode system (LDef) this winter / early spring; So we actually have the means to create a pure bytecode compiler and runtime.

But this is not a priority for us at this time. Smart Mobile Studio was made for JavaScript and while it would be cool to compile Delphi sourcecode to portable bytecodes, such a project would require not just a couple of namespaces – but a complete rewrite of the RTL. The assembler source-code and parser can be found in the “Next Generation Demos” folder (Smart Mobile Studio 3.0 demos). Feel free to build on the codebase if you fancy creating your own language;Get creative and enjoy! **Note: Using the assembler in your own projects requires a valid Smart Mobile license.

Native Apps

It’s interesting that people still ask this, since its one of our central advantages. We already generate native apps via the Phonegap post-processor.

phonegap

Phonegap turns your JS apps into native apps

Phonegap takes your compiled Smart (visual projects only) compiled code, processes it, and spits out native apps for every mobile OS on the market (and more). So you don’t have to compile especially for iOS, Android, Tizen or FireOS — Phonegap generates one for each system you need, ready for AppStore.

So we have native covered by proxy. And with millions of users Phonegap is not going anywhere.

Release date

We are going over the last beta as I type this, and Smart Mobile Studio 3.0 should be available next week. Which day is not easy to say, but at least before next weekend if all goes accoring to plan.

Make sure you visit www.smartmobilestudio.com and buy your license!

HTML5 Attributes, learn how to trigger conditional styling with Smart Mobile Studio

November 8, 2017 Leave a comment

Im not sure if I have written about Attributes before; Probably, because they are so awesome to work with. But today I’m going to show you something that makes it even more awesome, bordering on unbelievable.

What are HTML attributes again?

attribsBefore we dig into the juicy stuff, let’s talk about attributes. For those that dont know much about HTML or CSS, here is a quick and dirty overview. A lot of people use Smart Mobile Studio because they dont know CSS or HTML beyond the basics (or even because they dont want to learn it, quite a few cant stand JavaScript and CSS). Well that is not a problem.

Note-1: While not a vital prerequisite, I do suggest you buy a good book on JavaScript, HTML and CSS. If you are serious about using web technology (like node.js on the server) your Smart skills will benefit greatly by knowing how things work “under the hood” so to speak. You will make better Smart Mobile Studio applications and you will understand the RTL at a deeper level than the average user.

OK, back to attributes. You know how HTML tags have parameters right? For example, a link to another webpage looks like this:

<a href="http://blablabla">This is a link</a>

Note-2: I dont have time to teach you HTML from scratch, so if you have no idea what the “A” tag is then please google it.

Focus here is not on the “a” part, but rather on the “href” parameter. That is actually not a parameter but a tag-attribute (which must not be confused with a tag-property btw).

Back in the day attributes used to be exclusive; Meaning that if you tried to set some attribute value the tag didnt support – nothing would happen. The browser would just ignore it and the information would be deleted.

Around HTML4 all of that changed. Suddenly we got the freedom to declare our own attributes, regardless of tag. The only catch is that the attribute name must be prefixed with “data-“. Which makes sense because the browser needs to tell the difference between valid attributes, junk and intrinsic (supported) attributes.

Storing information outside the pascal instance

When you create a visual control, the control internally creates a DOM element (or tag object, same thing) that it manages. Most visual controls in our RTL manages a DIV element because that is just a square block that can be easily molded and shaped into whatever you like.

spjs_2105But, when you create a Smart Pascal class you dont just get a DOM element in return. You get a Smart Pascal object instance. This is the same as Delphi and Lazarus: a class is a blueprint of an object. You dont create classes you create instances.

The same thing happens when you use Smart Pascal: the JSVM (JavaScript virtual machine) delivers a JavaScript object instance – and that is what your code operates on. When you create a visual class instance, that in turn will create a DOM element and manage that until you release the Smart Pascal instance.

Storing information in a class is easy. It’s one of the fundamental aspects of object oriented programming and there really isnt that much to say about that. But what if you need to store information in a control you dont own? Perhaps you have installed a package you bought, or that a friend shared with you – and you cant change the class (or perhaps dont want to change the class). What then?

This is where the attribute object comes to the rescue. Because now you can store information directly in the DOM element rather than altering the class itself (!)

That is so powerful I dont even know where to start, because you can write libraries that can do amazing things without fields or demanding the user to change their controls (and in some cases, avoid forcing the user to inherit from a particular custom-control).

A real-life example

Our special effect unit, SmartCL.Effects.pas, uses this technique to keep track of effect state. When you execute an effect on a control a busy-flag is written as an attribute to the managed DOM object. And when the effect is finished the busy-flag is reset.

effects

Our CSS hardware powered effect unit uses attributes to keep track of running effects

If you execute 10 effects on a control, it’s this busy flag that stops all of them running at the same time (which would cause havoc). While this attribute is set any queued effects wait their turn.

This would be impossible to achieve without declaring a busy property, or doing some form of stacking behind the scene; both of them expensive codewise. But with attributes it’s a piece of cake.

And now for the juicy parts

styling-forms-with-cssNow that you know what attributes do and how awesome they are, what can possibly make them even more awesome? In short: “CSS attribute pseudo selectors” (phew, that is a mouthful isnt it!).

So what the heck is a pseudo selector? Again its a long story, so im just going to call it “states”. It allows you to define styles that should be activated when a particular state occurs. The most typical state is the :active state. When you press a button the DOM element is said to be active. This allows us to write CSS styles that are applied when you press the button (like changing the background, border or font-color).

But did you know you can also define styles that react to attribute changes?

Just stop and think about this for a moment:

  • You can define your own attributes
  • You can read, write and check for attributes
  • Attributes are part of the DOM element, not the JS instance
  • You can define CSS that apply when an element has an attribute
  • You can define CSS that apply if an attribute has a particular value

If you are still wondering what the heck this is good for, imagine the following:

With this, you can do the following:

  1. Write an event-handler for TW3Application.OnOrientationChange (an event that fires when the user rotate the mobile device horizontally or vertically).
  2. Store the orientation as a attribute value
  3. Define CSS especially for the orientation attribute values

The browser will automatically notice the attribute change and apply the corresponding CSS. This is probably one of the coolest CSS features ever.

Other things that come to mind:

  • You can write CSS that colors the rows in a grid or listbox based on the data-type the row contains. So an integer can have a different background from a float, boolean or string. And all of it can be automated with no code required on your part. You just need to write the CSS rule once and that’s it.
  • You can use attributes it to trigger pre-defined animations. In fact, you could pre define 100 different animations, and based on the attribute-name you can trigger the correct one. Again all of it can be neatly implemented as CSS.

Let’s make a button that triggers a style

While simple, the following example should serve as a good example. It’s easy to build on and not to complex. Let’s start with the CSS:

button[data-funky="this rocks"] {
  background: none;
  background-color: #FF00FF;
  font-color: #FFFFFF;
  font-size: 22px;
  font-weight: bold;
}

The CSS above should be easy to understand. First we define the name of the DOM element, which in my case is a button. Next we define the attribute, and like mentioned it has to be prefxed with “data-” (our attributes class does this automatically in the RTL, so you dont need to prefix it when you code). And finally we define the value the style should trigger on, “this rocks”.

Right, let’s write some code:

  var
  MyButton := TW3Button.Create(self);
  MyButton.SetBounds(100,280, 100, 44);
  MyButton.Caption := 'Click me!';
  MyButton.OnClick := procedure (Sender: TObject)
  begin
    var Text := MyButton.Attributes.Read('funky');
    if Text <> 'this rocks' then
      MyButton.Attributes.Write('funky','this rocks')
    else
      MyButton.Attributes.Write('funky','');
  end;

The code is very simple, we read the value of the attribute and then we do a toggle based on the content. So when you click the button it will just toggle the trigger value.

This is how the button looks before we click it:

normal

And when we click the button the attribute is written to, and it’s automatically styled:

styled

How cool is that! The things you can automate with this is almost endless. It is a huge boon for anyone writing mobile applications with Smart Mobile Studio and it makes what would otherwise be a difficult task ridiculously easy.

Cheers!

ClientRect, BoundsRect and adventures in Smart Pascal layout land

November 6, 2017 Leave a comment

HTML really is the kitchen sink of ideas. Some of them are good, others are bad – but all them have valid reason for being there.

When coming from Delphi or C++ builder to web development you really feel like you have tumbled down the rabbit hole from time to time. Especially when it comes to things like margins, padding and clientrect values.

You would imagine that BoundsRect gives you the full size of a control. In fact BoundsRect() should just be the same as putting left, top, width, height into a TRect structure right? Same with ClientRect, it should be the same as putting 0, 0, ClientWidth, ClientHeight into a TRect structure right?

Smart Mobile Studio uses absolute positioning, which means that you can layout controls at ordinary cartesian coordinate values. If you place a button at position 10, 10 – that means 10 pixels from the left edge and 10 pixels from the top edge. This is what we are used to from Delphi and other native languages.

But the browser have different boxing models, or box-sizing modes if you like. We are using the one best suited for per-pixel-positioning, namely “border-box”. This means that the width and height values for the control will include the size of the content, it’s padding and the size of the border. It excludes things like margin since that is just empty air the browser adds to the final co-ordinates of a visual control.

Doing it by the book

Since we are taking Smart out of the homebrew production style these days, there had to come a time where this must be dealt with.

If you don’t care about making your own controls then this wont effect you at all. You will always be able to drag & drop some controls on the form-designer, or (like most of us does) create them from code and perform layout during the Resize() method.

But .. if you want to make controls that conforms to our theme engine, that actually give a damn about margins, padding and wants give CSS the power to change existing controls the way it deserves, then you better pay attention.

Having experimented with this for a while now, here are the two cardinal rules you must follow if you want your controls to take height for the margin, padding and border-sizes defined in our CSS theme files:

  1. Margins only apply when positioning child elements with margins
  2. When doing layout of child elements, padding only apply from the parent or container of content, not the content itself.

Example for rule #1

Imagine you have a panel on a form. You want to populate that panel with 10 child elements and you want to do it properly, taking height for whatever padding the panel may have – and also whatever margin’s may exist for some child elements.

  var dx := W3Panel1.Border.Left.Padding;
  var dy := W3Panel1.Border.Top.Padding;
  for var x := 0 to 9 do
  begin
    var Item := FItems[x];
    var ItemRect := TRect.Create(dx, dy, Item.width, item. height);
    ItemRect.right -= (Item.Border.Margin.Left + Item.Border.Margin.Right);
    ItemRect.bottom -= (Item.Border.Margin.Top + Item.Border.Margin.Bottom);
    Item.SetBounds(ItemRect);
  end;

Look at the code above. Notice that we dont initialize dx and dy to 0 (zero). We could ofcourse but that would defeat the purpose of being CSS friendsly.

Also notice that we dont add the left and top margin to the final rectangle, this is because the browser automatically does this for us. Instead, we need to scale the right and bottom edge of the rectangle by subtracting the size of the left and right / top and bottom margins.

So if you want theme friendly layout’s, you have to go the extra mile and include these things.

Note: The above was just an example, our ClientRect() function already deals with padding for us, so you would set dx and dy to ClientRect.left and ClientRect.top.

ClientWidth and ClientHeight methods however, remain unaffected by padding. Because there will be cases where you want full control and non-conformity.

Example of rule #2

Think of a text-editor. You want to add a bit of margin to the document and simply drag the left-margin widget to where you need it. Padding for HTML elements works pretty much the same way.

To demonstrate I will create a test container class and a test child class.

First, create a new visual application to play with. Drop a TW3Panel control on the form and size it to full the form (with a bit of air from the edges naturally).

Next, go into project options and check the “use custom stylesheet”. That way Smart will clone whatever style you are using and create a new node in your project manager. Add the following CSS to the stylesheet:

.TTestOwner {
  margin: 20px;
  padding: 4px;
  border: 3px solid #FFFF00;
  background-color: #FF0000;
}

.TTestChild {
  margin: 10px;
  padding: 4px;
  border: 3px solid #000000;
  background-color: #FFFFFF;
}

With the CSS in place, add the following pascal classes to your mainform code, just below the “type” keyword:

TTestChild = class(TW3CustomControl)
end;

TTestOwner = class(TW3CustomControl)
end;

Now, prior to writing this article I make a couple of helper functions. If you are using Alpha 1 (which most of you are), add the following class and code to your form1 unit:

TW3Theme = class
public
  class function  AdjustRectToLayoutFactors(const ThisControl: TW3MovableControl; const Rect: TRect): TRect;
  class function  GetPaddedClientRect(const ThisControl: TW3MovableControl): TRect;
end;

class function TW3Theme.GetPaddedClientRect(const ThisControl: TW3MovableControl): TRect;
begin
  if ThisControl <> nil then
  begin
    result := TRect.Create(0, 0, ThisControl.ClientWidth, ThisControl.ClientHeight);
    result.Left += ThisControl.Border.Left.Padding;
    result.Top += ThisControl.Border.Top.Padding;
    result.Right -= ThisControl.Border.Right.Padding;
    result.Bottom -= ThisControl.Border.Bottom.Padding;
  end else
  result := TRect.NullRect;
end;

class function TW3Theme.AdjustRectToLayoutFactors(const ThisControl: TW3MovableControl; const Rect: TRect): TRect;
begin
  if ThisControl <> nil then
  begin
    (*  Rule #1, "margins should only be added when dealing with child elements"
        Since we are usins "border-box" as our size model, padding and border is
        already included in the clientwidth / clientheight values we get from the
        system.

        More importantly, margin only affect left and top edge of a rectangle because
        those are the only factors that affect MoveTo() type functionality in
        the browser itself.

        So when the browser moves an element to position 10px, 10px, it automatically
        adds the margin. If you have a margin of 10 pixels - the result will be
        (visually) that the control ends up at 20px, 20px instead. *)

    // Start with a carbon copy of the rectangle we were given
    result := Rect;

    (*  Note: The browser can only know about the left and top edge when
        placing elements. It cannot see into the future to know the exact
        height of an element, or if the content will suddenly grow. So we
        have to calculate the right and bottom based on our knowledge
        from the Rect parameter *)
    result.right  -= (ThisControl.border.left.margin + ThisControl.border.right.margin);
    result.bottom -= (ThisControl.border.top.margin + ThisControl.border.bottom.margin);

    (*  Rule #2: Padding should only be applied from a control's padding when
        calculating a position for that child. This is recursive, so a parent
        will apply this to their children, which each child will force it's
        padding on any children it may house. *)

    if ThisControl.parent <> nil then
    begin
      var Owner := TW3MovableControl(ThisControl.Parent);
      result.left += Owner.border.left.padding;
      result.top += owner.border.top.padding;
      result.right -= owner.border.right.padding;
      result.bottom -= owner.border.bottom.padding;
    end;
  end;
end;

With both styling and the pascal classes out-of-the-way, lets add some code to get the magic working.

So copy & paste this into your W3Form1.InitializeForm() procedure:

  var LRect:  TRect;
  var Box:    TTestOwner;
  var Child:  TTestChild;

  // Create parent container
  Box := TTestOwner.Create(W3Panel1);
  LRect := TRect.Create(0, 0, W3Panel1.ClientWidth, 300);
  LRect := TW3Theme.AdjustRectToLayoutFactors(Box, LRect);
  Box.SetBounds(LRect);

  // create child element for our test-owner
  Child := TTestChild.Create(Box);
  LRect := TRect.Create(0, 0, box.ClientWidth, box.ClientHeight);
  LRect := TW3Theme.AdjustRectToLayoutFactors(Child, LRect);
  Child.SetBounds(LRect);

Note: The TW3Theme class is a part of Alpha 2 which should hit the download section next week. But you should now have everything you need to get this working, no matter what version of Smart Mobile Studio you are using.

Putting it all together

OK, lets run our application and have a look at the results. What we should see is a panel on a form – inside that should be a box that is 20 pixels from the edges (since the CSS defines 20 pixel margins). The box also has 4 pixel padding defined, so the total offset from the edges should be 24 pixels.

The child control inside the box likewise has margin and padding. It operates with 10 pixels of margin and 4 pixels padding. It also sports a 3 pixel border. So let’s see what we have so far:

marginals

As you can see it’s not that hard to deal with; just a bit of a brain teaser. Those that write custom controls for Delphi are used to dealing with stuff like this all the time. The difference is that native languages are less cryptic about things, and they also make width / height return the full size of the control. Regardless of what the content may be.

You might have noticed that Delphi has a new “Align with margin” property? Not sure when it came into the system but somewhere around Delphi XE I believe (?). There you define the size of the margin – and Delphi does the rest. You don’t have to think about the size of the margin, and it only comes into play when the Align property is activated.

Final notes

We are doing some brainstorming on how to best deal with these things right now. Personally I think the code I have shown so far, especially the helper code, goes a long way to make this easy to work with.

Some have voiced that ClientRect should always start at zero, but why is that? Where does it say that Clientrect should always be “0,0, width-1, height-1” ? That is not the voice of reason, that is the sound of old habits! The whole point of having a ClientRect, be it in Delphi, Lazarus, C++ or C# is because this can change. It would be equally futile to demand that ClipRect should always be the same as client-rect. That is to utterly miss the whole point of sequential rendering and fast graphics.

So the lesson is: If you play by the rules and never use hard-coded values, then your code wont be affected. And if you want to adjust so your code is 100% theme compatible (and again, this only is valuable for component writers) then calling a simple function to get the rectangle adjusted for margin etc. is not exactly rocket science. It’s a one-liner.

Well, hope it helps!

Custom dialog and loading data from JSON in Smart Pascal

October 30, 2017 1 comment

Right now we are putting the finishing touches on our next update, which contains our new theme engine. As mentioned earlier (especially if you follow us on Facebook) the new system builds on the older – but we have separated border and background from the basic element styling.

When working with the new theme system, I needed an application that could demonstrate and show all the different border and background types, most of our visual controls – but also information about what Smart Mobile Studio is, what it’s features are and where you can buy it.

smarties_01

So it started as a personal application just to get a good overview of the CSS themes I was working on; but it has become an example in it’s own right.

Dont hardcode, just dont

If you look at the picture above, there is a MenuList with the options: “Introduction”, “Features” and “Where to buy”. When you click these I naturally want to inform the user about these things by displaying information.

I could have hardcoded the information text into the application; in many ways that would have been simpler (considering the data requirements here is practically insignificant). but all that text within the source? I hate mess like that.

Secondly, how exactly was I going to show this information? Would I use the modal framework already in place, or code something more lightweight?

As always I ended up making a new and more lightweight system. A reader style dialog appears and allows you to scroll vertically. The header contains the title of the information and a close button.

smarties_02

Typical “reader” style dialog with scrolling

I also used a block-box to prevent the user from reaching the UI until they click the close-button. You notice that the form, toolbar and header in the back is darkened. This is actually a control that is semi-transparent that does one thing: prevent anyone from clicking or interacting with the UI while the dialog is active.

The JSON file structure

json_structureThe structure I needed was very simple: our records would have a unique ID that we use to fetch and recognize content; It would also have a Title and Text property. It really doesnt have to be more difficult than that.

To work with the JSON i used the online JSON editor JSonEditorOnline, which is actually really good! It allows you to write your JSON and then format it so that special characters (like CR+LF) is properly encoded.

Putting it all together

Having coded the dialog thing first, I now sat down and finished a sort of “Turbo Pascal” record database system for this particular file format. It’s not very big nor extremely advanced – but that’s the entire point! Throwing in SQLite or MongoDB for something as simple as a few records of data – especially when the data is so simple, is just a complete waste of time and effort.

Right, let’s have a peek at the code shall we!

unit infodialog;

interface

uses
  System.Types,
  System.Types.Convert,
  System.Types.Graphics,
  System.Colors,
  System.JSON,

  SmartCL.Theme,
  SmartCL.FileUtils,
  SmartCL.System,
  SmartCL.Effects,
  SmartCL.Components,
  SmartCL.Scroll,
  SmartCL.Controls.Panel,
  SmartCL.Controls.Scrollbox,
  SmartCL.Controls.Header;

type

  TInfoDialog = class(TW3Panel)
  private
    FHead:    TW3HeaderControl;
    FBox:     TW3Scrollbox;
  protected
    procedure InitializeObject; override;
    procedure FinalizeObject; override;
    procedure Resize; override;
  public
    property  Header: TW3HeaderControl read FHead;
    property  Content: TW3Scrollbox read FBox;

    class function ShowDialog(Title, Content: string): TInfoDialog;
  end;

  TAppInfoRecord = record
    iiId:     string;
    iiTitle:  string;
    iiText:   string;
    procedure Clear;
    class function Create(const Id, Title, Text: string): TAppInfoRecord;
  end;

  TAppInfoDB = class(TObject)
  private
    FStack:     array of TStdCallback;
    FItems:     array of TAppInfoRecord;

    procedure   Parse(DBText: string);

    procedure   HandleDataLoaded(const FromUrl: string;
                const TextData: string; const Success: boolean);
  public
    property    Empty: boolean read ( (FItems.Count < 1) );
    property    Count: integer read (FItems.Count);
    property    Items[index: integer]: TAppInfoRecord
                read  (FItems[index])
                write (FItems[index] := Value);

    function    GetRecById(Id: string; var Info: TAppInfoRecord): boolean;

    procedure   LoadFrom(Url: string; const CB: TStdCallback);
    procedure   Clear;

    destructor  Destroy; override;
  end;


implementation

uses SmartCL.Application;

//#############################################################################
// TAppInfoRecord
//#############################################################################

class function TAppInfoRecord.Create(const Id, Title, Text: string): TAppInfoRecord;
begin
  result.iiId := id.trim();
  result.iiTitle := Title.trim();
  result.iiText := Text;
end;

procedure TAppInfoRecord.Clear;
begin
  iiId := '';
  iiTitle := '';
  iiText := '';
end;

//#############################################################################
// TAppInfoDB
//#############################################################################

destructor TAppInfoDB.Destroy;
begin
  if FItems.Count > 0 then
    Clear();
  inherited;
end;

procedure TAppInfoDB.Clear;
begin
  FItems.Clear();
end;

function TAppInfoDB.GetRecById(Id: string; var Info: TAppInfoRecord): boolean;
begin
  Info.Clear();
  if not Empty then
  begin
    Id := Id.trim().ToLower();
    if id.length > 0 then
    begin
      for var x := 0 to Count-1 do
      begin
        result := Items[x].iiId.ToLower() = Id;
        if result then
        begin
          Info := Items[x];
          break;
        end;
      end;
    end;
  end;
end;

procedure TAppInfoDB.Parse(DBText: string);
var
  vId:    variant;
  vTitle: variant;
  vText:  variant;
begin
  Clear();

  DbText := DbText.trim();
  if DbText.length > 0 then
  begin
    var FDb := TJSONObject.Create;
    FDb.FromJSON(DbText);

    if FDb.Exists('infotext') then
    begin
      // get the infotext-> [] array of JS objects
      var Root: TJSInstanceArray := TJSInstanceArray( FDb.Values['infotext'] );

      for var x := 0 to Root.Count-1 do
      begin
        var node := TJSONObject.Create(Root[x]);
        if node <> nil then
        begin
          Node
            .Read('id', vid)
            .Read('title', vtitle)
            .Read('text', vtext);

          FItems.add( TAppInfoRecord.Create(vId, vTitle, vText) );
        end;
      end;
    end;

  end;
end;

procedure TAppInfoDB.LoadFrom(Url: string; const CB: TStdCallback);
begin
  if assigned(CB) then
    FStack.push(CB);
  TW3Storage.LoadFile(Url, @HandleDataLoaded);
end;

procedure TAppInfoDB.HandleDataLoaded(const FromUrl: string;
          const TextData: string; const Success: boolean);
begin
  try
    // Parse if data ready
    if Success then
      Parse(TextData);
  finally
    // Perform callbacks
    while FStack.Count>0 do
    begin
      var CB := FStack.pop();
      if assigned(CB) then
        CB(Success);
    end;
  end;
end;

//#############################################################################
// TInfoDialog
//#############################################################################

procedure TInfoDialog.InitializeObject;
begin
  inherited;
  FHead := TW3HeaderControl.Create(self);
  FHead.BackButton.Visible := false;
  FHead.NextButton.Caption := 'Close';

  // By default the header text is centered within the space allocated for it,
  // which by default is 2/4. This can look a bit off when we never show
  // the left-button. So we force text-align to the left [normal].
  FHead.Title.Handle.style['text-align'] := 'left';

  FBox := TW3Scrollbox.Create(self);
  FBox.Background.FromColor(clWhite);
  FBox.ScrollBars := sbIndicator;
end;

procedure TInfoDialog.FinalizeObject;
begin
  FHead.free;
  FBox.free;
  inherited;
end;

procedure TInfoDialog.Resize;
begin
  inherited;
  var LBounds := ClientRect;
  var dy := LBounds.top;

  if FHead <> nil then
  begin
    FHead.SetBounds(LBounds.left, LBounds.top, LBounds.width, 32);
    inc(dy, FHead.Height +1);
  end;

  if FBox <> nil then
    FBox.SetBounds(LBounds.left, dy, LBounds.width, LBounds.height - dy);
end;

class function TInfoDialog.ShowDialog(Title, Content: string): TInfoDialog;
begin
  var Host := Application.Display;
  var Shade := TW3BlockBox.Create(Host);
  Shade.SizeToParent();

  var wd := Host.Width * 90 div 100;
  var hd := Host.Height * 80 div 100;
  var dx := (Host.Width div 2) - (wd div 2);
  var dy := (Host.Height div 2) - (hd div 2);

  var Dialog := TInfoDialog.Create(Shade);
  Dialog.Header.Title.Caption := Title;
  Dialog.SetBounds(dx, dy, wd, hd);
  Dialog.fxZoomIn(0.3, procedure ()
  begin
    Dialog.Content.Content.InnerHTML := Content;
    Dialog.Content.UpdateContent();
    Dialog.SetFocus();
  end);

  Dialog.Header.NextButton.OnClick := procedure (Sender: TObject)
  begin
    Dialog.fxFadeOut(0.2, procedure ()
    begin
      TW3Dispatch.Execute( procedure ()
      begin
        Dialog.free;
        Shade.free;
      end, 100);
    end);
  end;

  result := Dialog;
end;

end.

Using the code

The first thing you want to do is to create an instance of TAppInfoDb when your application starts. Remember to add your JSON file and that it’s formatted property, and then use the LoadFrom() method to load in the data:

  // Create our info database and load in the
  // introduction, features etc. JSON datafile
  FInfoDb := TAppInfoDB.Create;
  FInfoDb.LoadFrom('res/JSON1', nil);

The final parameter in the LoadFrom() method is a callback. So if you want to be notified when the file has loaded, just put an anonymous procedure there if you need it.

Showing a dialog with the information is then reduced to looking up the text you need by it’s ID, and firing up the reader dialog for it:

  W3Button1.OnClick := procedure (Sender: TObject)
  begin
    var LInfo: TAppInfoRecord;
    if FInfoDb.GetRecById('introduction', LInfo) then
      TInfoDialog.ShowDialog(LInfo.iiTitle, LInfo.iiText);
  end;

And that’s it! Simple, effective and ready to be dropped into any application. Enjoy!

Smart Mobile Studio and CSS: part 2

October 11, 2017 Leave a comment

In my previous article we had a quick look at some fundamental concepts regarding CSS. These concepts are not unique to Smart Mobile Studio, but simply just the way things work with CSS in general. The exception being the way Smart maps your pascal class-name to a CSS style of the same name.

To sum up what we covered last time:

  • Smart maps a control’s class-name to a CSS style with the same name. So if your control is called TMyControl, it expects to find a CSS style cleverly named “.TMyControl”. This works very well and is easy to apply.
  • CSS can affect elements recursively, so you can write CSS that changes the appearance and behavior of child controls. This technique is typically used if you inject HTML directly via the InnerHTML property
  • CSS is cascading, meaning that you can add multiple styles to the same control. The browser will merge them into a final, computed style. The rule of thumb is to avoid styles that affect the same properties
  • CSS can define more than colors; things like animations, gradients, animated gradients and whatnot can all be defined in CSS
  • Smart Mobile Studio ships with units for creating, applying and working with CSS from your pascal code. It also ships with effect classes that can trigger defined CSS animations.
  • Smart Mobile Studio has a special effect unit (SmartCL.Effects) that when added to the uses list, adds quite a few effect procedures to TW3MovableControl. These effect methods are prefixed with Fx (ex: fxMoveTo, fxFadeOut, fxScaleTo).

Best practices

When you write your own controls, don’t cheat. I have seen a lot of code where people create instances of TW3CustomControl for instance, and then jump through hoops trying to make that look good. TW3CustomControl is a base-class, it’s designed to be inherited from – not used “as is”. I can understand the confusion to some extent, I mean since TW3CustomControl manage a DIV by default – people with some HTML background probably think creating one of these is the same as making a DIV. But by doing so they essentially short-circuit the whole theme-system since (as underlined above) all pascal classes will use a style with the same name. And TW3CustomControl is just a transparent block of nothing.

No matter how small a thing you are creating, always inherit out your own classes and give them distinct names. This is extremely important with regards to styling, but also as a discipline of writing readable, maintainable code. Using TW3CustomControl all over the place will make the code a mess to maintain – let alone share with others who don’t have a clue what you are doing.

A practical example

To show how easy it is to style things once you have written code that uses distinct class names and clear-cut structure, let’s take the time to write a little list-box. Nothing fancy, just a control that can take X number of child rows, style them and display the items vertically like a list. Let’s begin with the class code:

type

  // Define an exception especially for our control
  EMyControl = class(EW3Exception);

  // Define a baseclass, that way we can grow in the future
  TMyChild = class(TW3CustomControl)
  end;

  // Define a class type, good when working with lists or
  // collections of elements that share ancestors
  TMyChildClass = class of TMyChild;

  // Define a clear child class, that way we can apply
  // styling without problems
  TMyChildRed = class(TMyChild)
  end;

  // Create a custom version with sensitive properties only
  // available to ancestors. Here we place these in the protected
  // section (items and count)
  TCustomMyControl = class(TW3CustomControl)
  protected
    property Items[const index: integer]: TMyChild
            read ( TMyChild(GetChildObject(Index)) ); default;
    property Count: integer read ( GetChildCount );

    procedure Resize; override;
  public
    function Add(const NewItem: TMyChild): TMyChild; overload;
    function Add(const &Type: TMyChildClass): TMyChild; overload;
  end;

  // The actual control we use, this is the one we write
  // CSS code for and that we create and use in our applications.
  // This step is optional ofcourse, but it has it's perks
  TMyControl = class(TCustomMyControl)
  public
    property Items;
    property Count;
  end;

Implementation

procedure TCustomMyControl.Resize;
var
  LCount: integer;
  bl, bt, br, bb: integer;
  wd, dy: integer;
  LItem: TMyChild;
begin
  inherited;

  // Avoid doing work if there is nothing there
  LCount := GetChildCount();
  if LCount > 0 then
  begin
    // Get the values of the borders/padding etc from CSS
    // We need to respect these when working in the client-rect
    bl := Border.Left.Width + Border.Left.Padding + Border.Left.Margin;
    bt := Border.Top.Width + Border.Top.Padding + Border.Top.Margin;
    br := Border.Right.Width + Border.Right.Padding + Border.Right.Margin;
    bb := Border.Bottom.Width + Border.Bottom.Padding + Border.Bottom.Margin;

    // This is the maximum width an element can have without
    // bleeding over whatever styling is present
    wd := ClientWidth - (bl + br);

    // Start at the top
    dy := bt;

    // Now layout each element vertically
    for var x := 0 to LCount-1 do
    begin
      LItem := Items[x];
      LItem.SetBounds(bl, dy, wd, LItem.Height);
      inc(dy, LItem.Height);
    end;
  end;
end;

function TCustomMyControl.Add(const &Type: TMyChildClass): TMyChild;
begin
  if &Type <> nil then
  begin
    // Start update
    BeginUpdate();

    // Create our control & return it
    result := &Type.Create(self);

      // Define that a resize must be issued
    AddToComponentState([csSized]);

    // End update. If update was not called elsewhere
    // the resize will happen now. If not, it will happen
    // when the last EndUpdate() is called (clever stuff!)
    EndUpdate();
  end else
  raise EMyControl.Create('Failed to add item, classtype was nil error');
end;

function TCustomMyControl.Add(const NewItem: TMyChild): TMyChild;
begin
  result := NewItem;
  if NewItem <> nil then
  begin
    // Are we the current parent?
    if not Handle.Contains(NewItem.Handle) then
    begin
      // Remove from other parent
      NewItem.RemoveFrom();

      // Start update
      BeginUpdate();

      // Add child to ourselves
      RegisterChild(NewItem);

      // Define that a resize must be issued
      AddToComponentState([csSized]);

      // End update. If update was not called elsewhere
      // the resize will happen now. If not, it will happen
      // when the last EndUpdate() is called (clever stuff!)
      EndUpdate();
    end;
  end else
  raise EMyControl.Create('Failed to add item, instance was nil error');
end;

If you are wondering about the strange property getter’s, where we don’t call a function but instead have some code inside (), that is another perk of Smart Pascal. The GetChildObject() method is a part of TW3TagContainer which ultimately TW3CustomControl inherits from, so we simply typecast and call that. This is perfectly legal in Smart as long as it’s a simple function or expression with matching type.

And now lets look at the CSS for our new control and its red child:

.TMyChildRed {
  padding: 2px;
  background-color: #FF0000;
  font-family: "Ubuntu", "Helvetica Neue", Helvetica, Verdana;
  color: #FFFFFF;
  border-bottom: 1px solid #AA0000;
}

.TMyControl {
  padding: 4px;
  background-color: #FFFFFF;
  border: 1px solid #000000;
  border-radius: 4px;
  margin: 1px;
}

We need to populate the list before we can see anything of course, so if we add the following code to InitializeForm() things will start to happen:

  // Lets create our control. We use an inline variable
  // here since this is just an example and I wont be
  // accessing it later. Otherwise you want to define it
  // as a form field in the form-class
  var LTemp := TMyControl.Create(self);
  LTemp.SetBounds(100, 100, 300, 245);

  // We call beginupdate here to prevent the
  // control calling Resize() for every elements.
  // It will only resize when the last EndUpdate (below)
  // is called. Also see how we use this inside the
  // procedures that needs to force a change
  LTemp.BeginUpdate();
  for var x := 1 to 10 do
  begin
    // Create a new "red" child
    var NewItem := LTemp.Add(TMyChildRed);

    // Fill the content with something
    NewItem.InnerHTML := 'Item number ' + x.ToString();
  end;
  LTemp.EndUpdate;

The end result might not look fancy but it demonstrates some very basic concepts that is fundamental to working with Smart Mobile Studio. Namely how to define CSS that map to your classes, and also how to use BeginUpdate() and EndUpdate() to prevent a ton of calls to Resize() when adding multiple items.

mycontrol

It wont win any prices for looks, but it demonstrates some very important principles when writing controls

Effects

Being able to style and layout child elements in your own controls is cool, but applications can quickly become dull and static without visual feedback. This is why I wrote the effect unit, namely to make it so easy to use GPU powered effects in your applications that anyone can make stuff move around.

So let’s make a little change to our mini-list control. When a user press one of the items, we want the item to scale up while the mouse is pressed, and then gracefully shrink back to normal size when you let go of the mouse. We could make it spin around for that matter, but let’s start with something a bit more down to earth.

This is where defining our own classes comes into play. We are going to add some code to our root child class, TW3MyChild, because this behavior should be universal. For sake of simplicity im just going to use the controls own events for this purpose. So Let’s expand our ancestor class to the following:

  TMyChild = class(TW3CustomControl)
  private
    FDown: boolean;
    procedure HandleMouseDown(Sender: TObject; Button: TMouseButton;
                        Shift: TShiftState; X, Y: integer);
    procedure HandleMouseUp(Sender: TObject; Button: TMouseButton;
                        Shift: TShiftState; X, Y: integer);
  protected
    procedure InitializeObject; override;
  end;

The implementation needs to keep track of when a scale is in process, otherwise we can scale the element out of sync with the UI. Again this is just an example, there are many ways to keep track of things but let’s keep it simple:

procedure TMyChild.InitializeObject;
begin
  inherited;
  self.OnMouseDown := HandleMouseDown;
  self.OnMouseUp := HandleMouseUp;
end;

procedure TMyChild.HandleMouseDown(Sender: TObject; Button: TMouseButton;
                    Shift: TShiftState; X, Y: integer);
begin
  if Button = TMouseButton.mbLeft then
  begin
    if not FDown then
    begin
      FDown := true;
      fxScaleUp(1.0, 1.5, 0.3);
    end;
  end;
end;

procedure TMyChild.HandleMouseUp(Sender: TObject; Button: TMouseButton;
                    Shift: TShiftState; X, Y: integer);
begin
  if Button=TMouseButton.mbLeft then
  begin
    if FDown then
    begin
      fxScaleDown(1.5, 1.0, 0.3, procedure ()
      begin
        FDown := false;
      end);
    end;
  end;
end;

The result? Well, when we press one of the items in our list that items grows to 1.5 of it’s original size (the parameter names for the effects are easy to understand). So we scale from 1.0 (normal size) to 1.5, and we tell the system to execute this transition in 0.3 seconds.

All the effect methods have an optional callback procedure you can use (anonymous procedure) that will fire when the effect is finished. As you can see in the HandleMouseUp() method we use this to reset the FDown field, allowing the effect to be executed again on the next click.

mycontrol2

Smooth scaling via hardware

Next time

Hopefully the past two articles have been interesting. In our next article we will look at some of the stuff we are building in our labs. That means talking about styling and how we are working to improving this (read: not yet available but in the process).

In the meantime, have a peek at what you can do with proper use of CSS effects

mycontrol3

You can do some amazing things with effects and JS (click image)

Happy coding!

Smart Pascal: Amibian vs. FriendOS

July 20, 2017 Leave a comment

This is not a new question, and despite my earlier post I still get hammered with these on a weekly basis – so lets dig into this subject and clean it up.

I fully understand that for non-developers suddenly having two Amiga like web desktops can be a bit confusing; especially since they superficially at least do many of the same things. But there is actually a lot of co-incidence surrounding all this, as well as evolution of the general topic. People who work with a topic will naturally come up with the same ideas from time to time.

But ok, lets dig into this and clear away any confusion

You know about FriendOS right? It looks a lot like Amibian

20100925-Designer

Custom native web servers has been a part of Delphi for ages, so it’s not that exciting for us

“A lot” is probably stretching it. But ok: ¬†FriendOS is a custom server system with a sexy desktop front-end written in HTML5. So you have a server that is custom written to interact with the browser in a special way. This might sound like a revolution to non-developers but it’s actually an established technology; its been a part of Delphi and C++ builder for at least 12 years now (Intraweb¬†being the best example, Raudus another). So if you are wondering why im not dazzled, it’s because this has been there for a while.

The whole point of Amibian.js is to demonstrate a different path; to get away from the native back-end and to make the whole system portable and platform independent. So in that regard the systems are diametrically different.

maxresdefault

Custom web servers that talk to your web-app is old news. Delphi developers have done this for a decade at least and it’s not really interesting at this point. Node.js holds much greater promise.

What FriendOS has done that is unique, and that I think is super cool – is to couple their server with RDP (remote desktop protocol) and some nice video streaming for smooth video chat. Again these are off the shelves parts that anyone can add once you have a native back-end, it’s not really hard to code but time-consuming; especially when you are potentially dealing with large number of users spawning threads all over the place. I think Friend-Labs have done an exceptional good job here.

When you combine these features it creates the impression of an operating system like environment. And this is perfectly fine for ordinary users. It all depends on your needs and what exactly you use the computer for.

And just to set the war-mongers straight:¬†FriendOS is not going up against Amibian. it’s going up against ChromeOS, Nayu and and a ton of similar systems; all of them with deep pockets and an established software portfolio. We focus on software development. Not even in the same ballpark.

To be perfectly frank: I see no real purpose for a web desktop except when connected to a context. There has to be an advantage beyond isolating web functions in one place. You need something special that your system does better than others, or different than others. Amibian has been about UAE.js and to run retro games in a familiar environment. And thus create a base that Amiga lovers can build on and play with. Again based on our prefab for customers that make embedded systems and use our compiler and RTL for that.

If you have a hardware product like a NAS, a ticket system or a retro-game machine and want to have a nice web front-end for it: then it makes sense. But there is absolutely nothing in both our systems that you can’t whip-up using Intraweb or Raudus in a few weeks. If you have the luxury of a native back-end, then adding Active Directory support is a matter of dropping a component. You can even share printers and USB devices over the wire if you like, this has been available to Delphi and c++ developers for ages. The “new” factor here, which FriendOS does very well i might add, is connectivity.

This might sound like criticism but it’s really not. It’s honesty and facts. They are going to need some serious cash to take on Google, Samsung, LG and various other players that have been doing similar things for a long time (or about to jump on the same concepts) — Amibian.js is for Amiga fans and people who use Smart Pascal to write embedded applications. We don’t see anything to compete with because Amibian is a prefab connected to a programming language. FriendOS is a unification system.

A programming language doesnt have the aspirations of a communication company. So the whole “oh who is best” or “are you the same” is just wrong.

Ok you say it’s not competing, but why not?

To understand Amibian.js you first need to understand Smart Pascal (see Wikipedia article on Smart Pascal). Smart Pascal (smartmobilestudio.com) is a software development studio for writing software using web technology rather than native machine-code. It allows you to create whatever you like, from games to servers, or kiosk software to the next Facebook clone.

Our focus is on enabling our customers to quickly program robust mobile applications, servers, kiosk software, games or large JavaScript projects; products that would otherwise be hard to manage if all you have is vanilla JavaScript. I mean why spend 2 years coding something when you can do it in 2 months using Smart? So a web desktop is just ridicules when you understand how large our codebase is and the scope of the product.

smart

Under Smart Pascal what people know as Amibian.js is just a project type. There is no competition between FriendOS and Amibian because a web desktop represents a ridicules small piece of our examples; it’s literally mistaking the car for the factory. Amibian is not our product, it is a small demo and prefab (pre fabricated system that others can download and build on) project that people use to save time. So under Smart, creating your own web desktop is a piece of cake, it’s a click, and then you can brand it, expand it and do whatever you like with it. Just like you would any project you create in Visual Studio, Delphi or C++ builder.

So we are not in competition with FriendOS because we create and deliver development tools. Our customers use Smart Pascal to create web environments both large and small, and naturally we deliver what they need. You could easily create a FriendOS clone in Smart if you got the skill, but again – that is but a tiny particle in our codebase.

Really? Amibian.js is just a project under Smart Pascal?

Indeed. Our product delivers a full object-oriented pascal compiler, debugger and IDE. So you can write classes, use inheritance and enjoy all the perks of a high-level language — and then compile this to JavaScript.

You can target node.js, the browser and about 90+ embedded devices out of the box. The whole point of Smart Pascal is to avoid the PITA that is writing large applications in JavaScript. And we do this by giving you a classical programming language that was made especially for application authoring, and then compile that to JavaScript instead.

Screenshot

Amibian.js is just a tiny, tiny part of what Smart Pascal is all about

This is a massive undertaking that started back in 2009/2010 and involves a high-quality compiler, linker, debugger and code generator; a full IDE with a ton of capabilities and last but not least: a huge run-time library that allows you to work with the DOM (document object model, or HTML) and node.js from the vantage point of a programmer.

Most people approach web development as a designer. They write html and then style them using a stylesheet. They work with colors, aspects and pages. Which means people who traditionally write programs falls between two chairs: first they must learn about html and css, and secondly a language which is ill equipped for large scale applications (imagine writing adobe photoshop in nothing but JS. Sure it’s possible, but wouldnt you rather spend a month coding that than a year? In a language that actually makes sense?).

With Smart you approach web development like you do writing programs. You work with visual controls, change properties, write code in response to events. Even writing your own visual controls that you can re-use and inherit from later is both fun and easy. So rather than ending up with a huge was of spaghetti code, which sadly is the fate of most large-scale JavaScript projects — Smart lets you work like you are used to. In a language better suited for the task.

And yes, I was not kidding when I said this was a huge undertaking. The source code in our codebase is close to 2.5 gigabytes. And keep in mind that this is source-code and libraries. So it’s not something you slap together over the weekend.

20108543_10154652373945906_5493167218129195758_n

The Smart source-code is close to 2.5 gigabytes. It has taken years to complete

But why do Amibian and FriendOS both focus on the Amiga?

That is pure co-incidence. The guys over at Friend Labs started out on the Amiga just like we did. So when I updated our desktop project (previously called Quartex Media Desktop) the Amiga look and feel came natural to me.

commodoreI’m a huge retro-computing fan that loves the Amiga. When I sat down to rewrite our window manager I loved the way Amiga OS 4.x looked, so I decided to implement an UI inspired by that.

People have to remember that the Amiga was a huge success in Scandinavia, so finding developers that are in their late 30s or early 40s that didn’t own an Amiga is harder than you think.

So the fact that we all root our ideas back to the Amiga is both co-incidence and a mutual passion for a great platform. One that really should have survived the financial onslaught of fat CEO’s and thir minions in the board.

But Amibian does a lot of what FriendOS does?

Probably. JavaScript is multi-tasking by default so if loading external URL’s into window containers, doing live resize and other things is what you refer to then yes. But that is the nature of web programming. Its like creating a bucket if you want to carry water; it is a natural first step of an evolutionary pattern. It’s not like FriendOS is copying us I would imagine.

240_F_61497799_GnuUiuJliH9AyOJTeo6i3bS8JNN7wgr2

For the record Smart started back in 2010 and the media desktop came in with the first hotfix, so its been available years before Friend-Labs even existed. Creating a desktop has not been a huge part of what we do because mobile applications, building a rich and solid run-time-library with hundreds of classes for our customers – and making an IDE that is great to use, that is our primary job.

We didn’t even know FriendOS existed. Let alone that it was a Norwegian product.

But you posted that you worked for FriendOS earlier?

Yes I did, very briefly. I was offered a position and I worked there for a month. It was a chance to work side by side with legends like David John Pleasance, ex head of Commodore for europe; and also my childhood hero Francois Lionet, author of Amos Basic for the Amiga way back in the 80’s and 90s.

blastfromthepast

We never forget our childhood heroes

Sadly we had our wires crossed. I am an awesome object pascal developer, while the guys at Friend-Labs are awesome C developers. I work primarily on Windows while they work mostly on Linux. So in essence they hired a Delphi developer to work in a language he doesn’t know on a platform he havent used.

They simply took for granted that I worked in C/C++, while I took for granted that they used object pascal. Its an easy mistake to make and its not the first time; and probably not the last.

Needless to say the learning curve would be extremely high for any developer (learning a new operating-system and programming language at the same time as you are supposed to be productive).

When my girlfriend suddenly faced a life threatening illness the situation became worse. It was impossible for me to commute or leave her side for the unforeseeable future; so when you add the six months learning curve to this situation; six months of not being able to contribute on the level I am used to; well I am old enough to know how that ends. So I did what was best for everyone and resigned.

Besides, I am a damn good Delphi developer with standing invitation to many companies; so it made more sense to just take a step backwards. Which was not fun because I really enjoyed the short time I was there. But, it was not meant to be.

And that is basically all there is to it.

Ok. But if Smart is a development tool, will it support Friend-OS ?

This is something that I really want to do. But since The Smart Company is a proper company with stocks, shareholders and investors – it’s not a decision I can take on my own. It is something that must be debated by the board. But personally yeah, I would love that.

friend

As they grow, so does the need for proper development tools

One of the reasons I hope FriendOS succeeds is because it’s a win-win situation. The more they expand the more relevant Smart becomes. Say what you will about JavaScript but writing large and complex applications is not easy by any measure.

So the moment we introduce Smart Pascal for Friend, their users will be able to write large applications rapidly, with better time-to-market and consequent ROI. So it’s a win-win. If they succeed then we get a bigger market; If they don’t we havent lost anything.

This may sound extremely self-serving, but Friend-Labs have had the same chance as everyone else to invest in Smart; our investor plans have been available for quite some time, and we have to do what is best for our company.

But what about Amibian, was it just a short thing?

Not at all. It is put on hold for a few months while we release the next generation RTL. Which is probably the biggest update in the history of Smart Pascal. We have a very clear agenda ahead of us and Amibian.js is (as underlined) a very small part of what we do.

But Amibian is written using our next generation RTL, and without that our customers cant really do much with it. So it’s important to get the RTL out first and then work on the IDE to reflect its many new features. After that – Amibian.js development will continue.

The primary target for Amibian.js is embedded devices and kiosk systems, coupled with full-screen web applications and hardware front-ends (NAS and backup devices being great examples). So the desktop will run on affordable, off the shelves hardware starting at $40 and all the way up to the most powerful and expensive x86 boards on the market. Cheap solutions like Raspberry PI, ODroid XU4 and Tinkerboard will deliver what you today need a dedicated $120 x86 board to achieve.

kiosk-systems

Our desktop will run on many targets and is platform independent by design

This means that our deskop has a wildly different modus operandi. We will not require a constant connection to a remote server. Amibian will happily boot up on a single device, regardless of processor type.

Had we coded our backend using Delphi or C++ builder (native like FriendOS have done) we would have been finished months ago. And I could have caught up with FriendOS in a couple of months if I wanted to. But that is not in our agenda. We have written our server framework for node.js as we coded the desktop ¬†– which means it’s platform and OS agnostic by design. If node.js runs, Amibian will run. It wont care if you are running on a $40 embedded board or the latest Intel i9 cpu.

Last words

I really hope this has helped and that the confusion between Amibian.js and our agenda, versus what Friend-Labs is doing, is now clearer.

Amibian666

From Norway with love

I wish Friend-Labs the very best and hope they are successful in their endeavour. They have worked very hard on the product and deserve that. And while I might come over as arrogant at times, im really not.

Web desktops have been around for a long time now (Asustor is my favorite) through Delphi and C++ builder and that is just facts. But that doesn’t mean you can’t put things together in new and interesting ways! Smart itself was first put together by existing technology. It was said to be impossible by many because JavaScript and object pascal are unthinkable companions. But it turned out to be a perfect match.

As for the future – personally I don’t believe in the web-desktop outside a specific context, something to give it purpose if you like. I believe for instance that Amibian.js will be awesome for Amiga users when its running on a $99 ARM laptop. Where the system boots straight into a full-screen desktop and where UAE.js is fully integrated into the core, making retro-gaming and running old programs close to seamless. That I can believe in.

But it would make no sense running Amibian or FriendOS in a browser on top of a Windows desktop or a full Ubuntu X session. Unless the virtual desktop functions as your corporate window with access to company mail, documents and essentially what every web-based intranet already does. So once again we end up with the fact that this has already been done. And unless you create a unique context for it, it just wont have any appeal.¬†This is also why I havent pursued¬†the same tech Friend-Labs have, because that’s not where the exciting stuff is happening.

But I will happily be proven wrong, because that means an even bigger market for us should we decide to support the platform.

Smart Desktop: Inter frame communication

April 24, 2017 Leave a comment

What is a process in a web desktop environment? Been thinking a bit about this, and it’s actually not that easy to hang the reality of an external script or website running in a frame as a solid “concept” (I use the word entity in my research notes). Visually it appears as a secondary, isolated web application – but ultimately (or technically) it’s just another web object with its own rendering context.

amibian_cortanaWhat I’m talking about is programs. Amibian.js or Smart desktop can already run a wealth of applications – which ultimately are just external or internal web-pages (a very simple term in some cases, but it will have to do). But when you want complex behavior from an eco-system, you must also provide an equally complex means of communication.

And this brings me neatly to the heart of todays task – namely how the desktop can talk to running web-applications and visa versa. And there are ultimately only two real options to choose from here: pure client, or client-server.

Unlike other embedded / web desktops I want to do as much as possible client-side. It would take me five minutes to implement message dispatching if I did this server-side, but the less dependent we are on a server – the more flexible amibian.js will be in terms of integration.

Screenshot

The first Desktop-Application type running in the desktop! And its aware that it is running under Amibian. This frame/host awareness helps establish parent/child relationship

Turns out that the browser has an API for this. A message API that allows a main website to talk with other websites embedded in IFrames. Which is just what we need.

But its very crude and not as straight forward as you may think. You have commands like postMessage() and then you can set up an event-handler that fires whenever something comes through. But you also have security measures to think about. What if you interface with a web-application designed to kill the system? Without the safety of an abstraction layer that would indeed be a very easy hack.

The Smart message API

This has been in the RTL for quite some time but I havent really used it for much. I had one demo a couple of years back that used the message-api to handle off-screen rendering in a separate IFrame (sort of poor mans thread), but that was it.

Well, time to update and bring it into 2017! So I give you the new and improved inter-frame, inter-window and inter-domain message framework! (ta da!)

It should be self-evident how to use it, but a little context wont hurt:

  • You inherit from TW3MessageData and implement whatever data-fields you need
  • You send messages using the w3_SendMessage() method
  • You broadcast (i.e send the message to all frames and windows listening) messages using the w3_broadcastMessage method

But here comes the twist, because I hate having to write a bunch of if/then/else switches for messages. So instead I created a simple subscription system! Super small yet very efficient!

msgbroker

In other words, you inform the system which messages you support, which then becomes your subscription. Whenever the local dispatcher encounters that particular message type  Рit will forward the message object to you.

Here is the code. Notice the helper functions that deals with finding the current domain and so on – this will be handy when establishing a parent/child relationship.

Also note: you are expected to inherit from these classes and shape them to your message types. I have implemented the bare-bones that mimic’s Windows message API. And yes, please google inter-frame communication while testing this.

unit SmartCL.Messages;

interface

uses
  w3c.dom,
  System.Types,
  System.Time,
  System.JSON,
  Smartcl.Time,
  SmartCL.System;

const
  CNT_QTX_MESSAGES_BASEID = 1000;

type

  TW3MessageData         = class;
  TW3CustomMsgPort       = class;
  TW3OwnedMsgPort        = class;
  TW3MsgPort             = class;
  TW3MainMessagePort     = class;
  TW3MessageSubscription = class;

  TW3MessageSubCallback  = procedure (Message: TW3MessageData);
  TW3MsgPortMessageEvent = procedure (Sender: TObject; EventObj: JMessageEvent);
  TW3MsgPortErrorEvent   = procedure (Sender: TObject; EventObj: JDOMError);

  (* The TW3MessageData represents the actual message-data which is sent
     internally by the system. Notice that it inherits from JObject, which
     means it supports 1:1 mapping from JSON.
     This is why Deserialize is a member, and Serialize is a class member. *)
  TW3MessageData = class(JObject)
  public
    property    ID: Integer;
    property    Source: String;
    property    Data: String;

    function Deserialize: string;
    class function Serialize(const ObjText: string): TW3MessageData;

    constructor Create;
  end;

  (* A "mesage port" is a wrapper around the [obj]->Window[]->OnMessage
     event and [obj]->Window[]->PostMessage API.
     Where [obj] can be either the main window, or an embedded
     IFrame->contentWindow. This base-class implements the generic behavior.
     You must inherit or use a decendant class if you dont know exactly what
     you are doing! *)
  TW3CustomMsgPort = Class(TObject)
  private
    FWindow:    THandle;
  protected
    procedure   Releasewindow;virtual;
    procedure   HandleMessageReceived(eventObj: JMessageEvent);
    procedure   HandleError(eventObj: JDOMError);
  public
    Property    Handle:THandle read FWindow;
    Procedure   PostMessage(msg:Variant;targetOrigin:String);virtual;
    procedure   BroadcastMessage(msg:Variant;targetOrigin:String);virtual;
    Constructor Create(WND: THandle);virtual;
    Destructor  Destroy;Override;
  published
    property    OnMessageReceived: TW3MsgPortMessageEvent;
    Property    OnError: TW3MsgPortErrorEvent;
  end;

  (* This is a baseclass for window-handles that already exist.
     You are expected to provide that handle via the constructor *)
  TW3OwnedMsgPort = Class(TW3CustomMsgPort)
  end;

  (* This message-port type is very different from both the
     ad-hoc baseclass and "owned" variation above.
     This one actually creates it's own IFrame instance, which
     means it's a completely stand-alone entity which doesnt need an
     existing window to dispatch and handle messages *)
  TW3MsgPort = Class(TW3CustomMsgPort)
  private
    FFrame:     THandle;
    function    AllocIFrame: THandle;
    procedure   ReleaseIFrame(aHandle: THandle);
  protected
    procedure   ReleaseWindow; override;
  public
    constructor Create; reintroduce; virtual;
  end;

  (* This message port represent the "main" message port for any
     application that includes this unit. It will connect to the main
     window (deriving from the "owned" base class) and has a custom
     message-handler which dispatches messages to any subscribers *)
  TW3MainMessagePort = Class(TW3OwnedMsgPort)
  protected
    procedure HandleMessage(Sender: TObject; EventObj: JMessageEvent);
  public
    constructor Create(WND: THandle); override;
  end;

  (* Information about registered subscriptions *)
  TW3SubscriptionInfo = Record
    MSGID:    integer;
    Callback: TW3MessageSubCallback;
  end;

  (* A message subscription is an object that allows you to install
     X number of event-handlers for messages you want to recieve. Its
     important to note that all subscribers to a message will get the
     same message -- there is no blocking or ownership concepts
     involved. This system is a huge improvement over the older WinAPI *)
  TW3MessageSubscription = class(TObject)
  private
    FObjects:   Array of TW3SubscriptionInfo;
  public
    function    ValidateMessageSource(FromURL: string): boolean; virtual;
    function    SubscribesToMessage(const MSGID: integer): boolean; virtual;
    procedure   Dispatch(const Message: TW3MessageData); virtual;
    function    Subscribe(const MSGID: integer; const CB: TW3MessageSubCallback): THandle;
    procedure   Unsubscribe(const Handle: THandle);

    Constructor Create; virtual;
    Destructor  Destroy; override;
  end;

  (* Helper functions which simplify message handling *)
  //function  W3_MakeMsgData:TW3MessageData;
  procedure W3_PostMessage(const msgValue: TW3MessageData);
  procedure W3_BroadcastMessage(const msgValue: TW3MessageData);

  (* Audience returns true if a message-ID have any
     subscriptions assigned to it *)
  function  W3_Audience(msgId: integer): boolean;

  (* This returns true if the current smart application is embedded in a frame.
     That can be handy to know when establishing parent/child relationships
     between main-application and child "programs" running in frames *)
  function AppRunningInFrame: boolean;
  function GetUrlLocation: string;
  function GetDomainFromUrl(URL: string; const IncludeSubDomain: boolean): string;

implementation

uses SmartCL.System;

var
_mainport:    TW3MainMessagePort = NIL;
_subscribers: Array of TW3MessageSubscription;

function AppRunningInFrame: boolean;
var
  LMyFrame: THandle;
begin
  // window.frameElement Returns the element (such as <iframe> or <object>)
  // in which the window is embedded, or null if the window is top-level.
  try
    // Note: attempting to read frameElement will throw a SecurityError
    // exception in cross-origin iframes. It should just return null,
    // but not all browsers play by the rules -- hence the try/catch
    asm
      @LMyFrame = window.frameElement;
    end;
  except
    on e: exception do
    exit;
  end;
  result := not (LMyFrame);
end;

function GetUrlLocation: string;
begin
  asm
    @result = window.location.hostname;
  end;
end;

function GetDomainFromUrl(Url: string; const IncludeSubDomain: boolean): string;
begin
  url := Url.trim();
  if url.length < 1 then
  begin
    asm
      @Url = window.location.hostname;
    end;
  end;
  asm
    @url = (@url).replace("/(https?:\/\/)?(www.)?/i", '');
    if (!@IncludeSubDomain) {
        @url = (@url).split('.');
        @url = (@url).slice((@url).length - 2).join('.');
    }
    if ((@url).indexOf('/') !== -1) {
      @result = (@url).split('/')[0];
    } else {
      @result = @url;
    }
  end;
end;

//#############################################################################
//
//#############################################################################

procedure QTXDefaultMessageHandler(sender:TObject;EventObj:JMessageEvent);
var
  x:      Integer;
  mItem:  TW3MessageSubscription;
  mData:  TW3MessageData;
begin
  mData := TW3MessageData.Serialize(EventObj.Data);
  //mData := new TW3MessageData();
  //mData.Serialize(EventObj.Data);

  for x:=0 to _subscribers.count-1 do
  Begin
    mItem := _subscribers[x];

    // Check that the message is subscribed to
    if mItem.SubscribesToMessage(mData.ID) then
    begin
      // Make sure subscriber validates the caller-domain
      if mItem.ValidateMessageSource( TVariant.AsString(EventObj.source) ) then
      begin
        (* We execute with a minor delay, allowing the browser to
         exit the function before we dispatch our data *)
        TW3Dispatch.Execute(procedure ()
        begin
          mItem.Dispatch(mData);
        end, 10);
      end;
    end;
  end;
end;

{ function W3_MakeMsgData: TW3MessageData;
begin
  result := new TW3MessageData();
  result.Source := "*";
end;      }

function W3_GetMsgPort: TW3MainMessagePort;
begin
  if _mainport = NIL then
  _mainport := TW3MainMessagePort.Create(BrowserAPI.Window);
  result := _mainport;
end;

procedure W3_PostMessage(const msgValue:TW3MessageData);
begin
  if msgValue<>NIL then
  W3_GetMsgPort().PostMessage(msgValue.Deserialize,msgValue.Source) else
  raise exception.create('Postmessage failed, message object was NIL error');
end;

procedure W3_BroadcastMessage(const msgValue:TW3MessageData);
Begin
  if msgValue<>NIL then
  W3_GetMsgPort().BroadcastMessage(msgValue,msgValue.Source) else
  raise exception.create('Broadcastmessage failed, message object was NIL error');
end;

function  W3_Audience(msgId: Integer): boolean;
var
  x:  Integer;
  mItem:  TW3MessageSubscription;
begin
  result:=False;
  for x:=0 to _subscribers.count-1 do
  Begin
    mItem:=_subscribers[x];
    result:=mItem.SubscribesToMessage(msgId);
    if result then
    break;
  end;
end;

//#############################################################################
// TW3MainMessagePort
//#############################################################################

Constructor TW3MessageData.Create;
begin
  self.Source:="*";
end;

function  TW3MessageData.Deserialize: string;
begin
  result := JSON.Stringify(self);
end;

class function TW3MessageData.Serialize(const ObjText: string): TW3MessageData;
begin
  result := TW3MessageData(JSON.parse(ObjText));
end;

//#############################################################################
// TW3MainMessagePort
//#############################################################################

constructor TW3MainMessagePort.Create(WND: THandle);
begin
  inherited Create(WND);
  OnMessageReceived := QTXDefaultMessageHandler;
end;

procedure TW3MainMessagePort.HandleMessage(Sender: TObject; EventObj: JMessageEvent);
begin
  QTXDefaultMessageHandler(self, eventObj);
end;

//#############################################################################
// TW3MessageSubscription
//#############################################################################

Constructor TW3MessageSubscription.Create;
begin
  inherited Create;
  _subscribers.add(self);
end;

Destructor TW3MessageSubscription.Destroy;
Begin
  _subscribers.Remove(self);
  inherited;
end;

function TW3MessageSubscription.Subscribe(const MSGID: integer; const CB: TW3MessageSubCallback): THandle;
var
  LObj: TW3SubscriptionInfo;
begin
  LObj.MSGID := MSGID;
  LObj.Callback := @CB;
  FObjects.add(LObj);
  result := LObj;
end;

procedure TW3MessageSubscription.Unsubscribe(const Handle: THandle);
var
  x:  Integer;
begin
  for x:=0 to FObjects.Count-1 do
  begin
    if variant(FObjects[x]) = Handle then
    Begin
      FObjects.delete(x,1);
      break;
    end;
  end;
end;

function TW3MessageSubscription.ValidateMessageSource(FromURL: string): boolean;
begin
  // by default we accept messages from anywhere
  result := true;
end;

function TW3MessageSubscription.SubscribesToMessage(const MSGID: integer): boolean;
var
  x:  Integer;
begin
  result:=False;
  for x:=0 to FObjects.Count-1 do
  Begin
    if FObjects[x].MSGID = MSGID then
    Begin
      result:=true;
      break;
    end;
  end;
end;

procedure TW3MessageSubscription.Dispatch(const Message:TW3MessageData);
var
  x:  Integer;
begin
  for x:=0 to FObjects.Count-1 do
  Begin
    if FObjects[x].MSGID = Message.ID then
    Begin
      if assigned(FObjects[x].Callback) then
      FObjects[x].Callback(Message);
      break;
    end;
  end;
end;

//#############################################################################
// TW3MsgPort
//#############################################################################

Constructor TW3MsgPort.Create;
begin
  FFrame:=allocIFrame;
  if (FFrame) then
  inherited Create(FFrame.contentWindow) else
  Raise Exception.Create('Failed to create message-port error');
end;

procedure TW3MsgPort.ReleaseWindow;
Begin
  ReleaseIFrame(FFrame);
  FFrame:=unassigned;
  Inherited;
end;

Procedure TW3MsgPort.ReleaseIFrame(aHandle:THandle);
begin
  If (aHandle) then
  Begin
    asm
      document.body.removeChild(@aHandle);
    end;
  end;
end;

function TW3MsgPort.AllocIFrame: THandle;
Begin
  asm
    @result = document.createElement('iframe');
  end;

  if (result) then
  begin
    /* if no style property is created, we provide that */
    if not (result['style']) then
    result['style'] := TVariant.createObject;

    /* Set visible style to hidden */
    result['style'].display := 'none';

    asm
      document.body.appendChild(@result);
    end;

  end;
end;

//#############################################################################
// TW3CustomMsgPort
//#############################################################################

Constructor TW3CustomMsgPort.Create(WND: THandle);
Begin
  inherited Create;
  FWindow := WND;
  if (FWindow) then
  begin
    FWindow.addEventListener('message', @HandleMessageReceived, false);
    FWindow.addEventListener('error', @HandleError, false);
  end;
End;

Destructor TW3CustomMsgPort.Destroy;
begin
  if (FWindow) then
  begin
    FWindow.removeEventListener('message', @HandleMessageReceived, false);
    FWindow.removeEventListener('error', @HandleError, false);
    ReleaseWindow;
  end;
  inherited;
end;

procedure TW3CustomMsgPort.HandleMessageReceived(eventObj:JMessageEvent);
Begin
  if assigned(OnMessageReceived) then
  OnMessageReceived(self,eventObj);
End;

procedure TW3CustomMsgPort.HandleError(eventObj:JDOMError);
Begin
  if assigned(OnError) then
  OnError(self,eventObj);
end;

procedure TW3CustomMsgPort.Releasewindow;
begin
  FWindow := Unassigned;
end;

Procedure TW3CustomMsgPort.PostMessage(msg:Variant;targetOrigin:String);
begin
  if (FWindow) then
  FWindow.postMessage(msg,targetOrigin);
end;

// This will send a message to all frames
procedure TW3CustomMsgPort.BroadcastMessage(msg:Variant;targetOrigin:String);
var
  x:  Integer;
  mLen: Integer;
begin
  mLen := TVariant.AsInteger(browserAPI.Window.frames.length);
  for x:=0 to mLen-1 do
  begin
    browserAPI.window.frames[x].postMessage(msg,targetOrigin);
  end;
end;

finalization
begin
  if assigned(_mainport) then
  _mainport.free;
end;

end.

Smart Pascal + WebOS = true

April 15, 2017 Leave a comment

LG-WebOSIf you own a television (and who doesn’t) chances are you own an LG model. The LG brand has been on the market since before recorded history (or so it feels) and have been a major player for decades.

What is little known though, is that LG also owns and finance a unique operating system. This is not uncommon these days, most NAS and cloud vendors have their own system (there are roughly 20 cloud based systems on the market that I know of). The only way to make a Smart TV (sigh) is to add a computer to it. And if you buy a television today chances are it contains a small embedded board running some custom-made operating system designed to do just that.

Every vendor has their own system, and those that don’t usually end up forking Android and adapt that to their needs.

Luna WebOS

LG’s system is called WebOS. This may sound like yet another html eye candy front end, but hear me out because this OS¬†is not what it seems.

VirtualBox_Developers LuneOS emulator appliance 20151006131924-stable-038-253_15_04_2017_16_01_43

Except for some font issues (see disk label) Smart loaded fine under WebOS

Remember Palm OS? If you are between 35 and 45 you should remember that before Apple utterly demolished the mobile-phone market with their iPhone, one of the most popular brands was Palm. Their core product being “Palm Pilot”, a kind of digital¬†filo-fax¬†(personal organizer) and mobile phone rolled into one.

WebOS is based on what used to be called Palm-OS, but it has been completely revamped, given a sexy new user interface (looks a lot like android to be honest!) and brought into the present age. And best of all: its 100% free! It runs on a plethora of systems, from x86 to Raspberry PI to Mips. It is used in televisions, terminals and set-top-boxes around the world – and is quite popular amongst engineers.

Check it out their new portal here; there is also plenty of links to pre-built images if you look around there: https://pivotce.com/

JavaScript application stack

One of the coolest features in my view, is that their applications are primarily made by JavaScript. The OS itself is native of course, but they have discovered that JavaScript and HTML5 is an excellent way to build applications. Applications that is easy to control, sandboxed and safe yet incredibly powerful and capable.

VirtualBox_Developers LuneOS emulator appliance 20151006131924-stable-038-253_15_04_2017_16_00_46

Smart Desktop booting Quake 3 in Luna OS

Well, today I sent them an Email requesting their SDK. I know they use Enyo.js as their primary and suggested framework – but that is no problem for Smart Mobile Studio. A better question is: can Luna handle our codebase?

When I loaded up Quake 3 in pure Asm.JS the TV crashed with a spectacular access violation. So they are probably not used to the level of hardcore coding Smart developers represent. But yeah, Quake III is about as advanced as it gets – and it pushes any browser to the outer limits of what is possible.

Once I get the SDK, docs and a few examples – I will begin adding support for it as quickly as possible. Which means you get a new project type especially for the platform + API units (typically stored under $RTL\Apis folder).

Why is this cool?

Because with support for Luna / WebOS, you as a Delphi or Smart developer can offer your services to companies that use WebOS or Luna (the open source version) in their devices. The list of companies is substantial – and they are well established companies. And as you have no doubt noticed, hardware engineers doesn’t always make the best software engineers. So this opens up plenty of opportunities for a good object pascal / Smart developer.

17948597_10154372538120906_1912877833_o

Luna has an Android like quality over it – except its more smooth to use

Remember Рother developers will use vanilla JavaScript. You have the onslaught of the VJL and the might of our libraries at your disposal. You can produce in days what others use weeks on achieving.

These are exciting days! I’ll keep you posted on our progress!

Smart Pascal, the next generation

April 15, 2017 1 comment

I want to take the time to talk a bit about the future, because like all production companies we are all working towards lesser and greater goals. If you don’t have a goal then you are in trouble; Thankfully our goals have been very clear from the beginning. Although I must admit that our way there has been.. “colorful” at times.

When we started back in 2010 we didn’t really know what would become of our plans. We only knew that this was important; there was a sense of urgency and “we have to build this” in the air; I think everyone involved felt that this was the case, without any rational explanation as to why. Like all products of passion it can consume you in a way – and you work day and night on turning an idea into something real. From the intangible to the tangible.

transitions_callback

It seems like yesterday, but it was 5 years ago!

By the end of 2011 / early 2012, Eric and myself had pretty much proven that this could be done. At the time there were more than enough nay-sayers and I think both of us got flamed quite often for daring to think different. People would scoff at me and say I was insane to even contemplate that object pascal could ever be implemented for something as insignificant and mediocre as JavaScript. This was my first meeting with a sub-culture of the Delphi and C++ community, a constellation I have gone head-to-head with on many occasions. But they have never managed to shake my resolve as much as inch.

 

 

When we released version 1.0 in 2012 some ideas about what could be possible started to form. J√łrn¬†defined plans for a system we later dubbed “Smart net”. In essence it would be something you logged onto from the IDE – allowing you to store your projects in the cloud, compile in the cloud (connected with Adobe build services) and essentially¬†move parts of your eco-system to the cloud. Keep in mind this was when people still associated cloud with “storage”; they had not yet seen things like Uber or Netflix¬†or played Quake 3 at 160 frames per second,¬†courtesy of asm.js in their browser.

The second part would be a website where you could do the same, including a live editor, access to the compiler and also the ability to buy and sell components, solutions and products. But for that we needed a desktop environment (which is where the Quartex Media Desktop came in later).

cool

The first version of the Media Desktop, small but powerful. Here running in touch-screen mode with classical mobile device layout (full screen forms).

Well, we have hit many bumps along the road since then. But I must be honest and say, some of our detours have also been the most valuable. Had it not been for the often absurd (to the person looking in) research and demo escapades I did, the RTL wouldn’t be half as powerful as it is today. It would deliver the basics, perhaps piggyback on Ext.js or some lame, run of the mill¬†framework – and that would be that. Boring, flat and limited.

What we really wanted to deliver was a platform. Not just a website, but a rich environment for creating, delivering and enjoying web and cloud based applications. And without much fanfare – that is ultimately what the Smart Desktop and it’s sexy node.js back-end is all about is all about.

We have many project types in the pipeline, but the Smart Desktop type is by far the most interesting and powerful. And its 100% under your control. You can create both the desktop itself as a project – and also applications that should run on that desktop as separate projects.

This is perfectly suited for NAS design (network active storage devices can usually be accessed through a web portal on the device), embedded boards, intranets and even intranets for that matter.

You get to enjoy all the perks of a multi-user desktop, one capable of both remote desktop access, telnet access, sharing files and media, playing music and video (we even compiled the mp4 codec from C to JavaScript so you can play mp4 movies without the need for a server backend).

The Smart Desktop

The Smart Desktop project is not just for fun and games. We have big plans for it. And once its solid and complete (we are closing in on 46% done), my next side project will not be more emulators or demos Рit will be to move our compiler(s) to Amazon, and write the IDE itself from scratch in Smart Pascal.

smart desktop

The Smart Desktop – A full desktop in the true sense of the word

And yeah, we have plans for EmScripten as well – which takes C/C++ and compiles it into asm.js. It will take a herculean effort to merge our RTL with their sandboxed infrastructure –¬†but the benefits are too great to ignore.

As a bonus you get to run native 68k applications (read: Amiga applications) via emulation. While I realize this will be mostly interesting for people that grew up with that machine – it is still a testament to the power of Smart and how much you can do if you really put your mind to it.

Naturally, the native IDE wont vanish. We have a few new directions we are investigating here – but native will absolutely not go anywhere. But cloud, the desktop system we are creating, is starting to become what we set out to make five years ago (has it been half a decade already? Tempus fugit!). As you all know it was initially designed as an example of how you could write full-screen applications for Raspberry PI and similar embedded devices. But now its a full platform in its own right – with a Linux core and node.js heart, there really is very little you cannot do here.

scsc

The Smart Pascal compiler is one of our tools that is ready for cloud-i-fication

Being able to login to the Smart company servers, fire up the IDE and just code – no matter if you are: be it Spain, Italy, Egypt, China or good old USA¬†— is a pretty awesome thing!

Clicking compile and the server does the grunt work and you can test your apps live in a virtual window; switch between device layouts and targets — then hit “publish” and it goes to Cordova (or Delphi) and voila – you get a message back when binaries for 9 mobile devices is ready for deployment. One click to publish your applications on Appstore, Google play and Microsoft marketplace.

Object pascal works

People may have brushed off object pascal (and from experience those people have a very narrow view of what object pascal is all about), but when they see what Smart delivers, which in itself is written in Delphi, powered by Delphi and should be in every Delphi developer’s toolbox — i think it should draw attention to both Delphi as a product, object pascal as a language – and¬†smart as a solution.

With Smart it doesn’t matter what computer you use. You can sit at home with the new A1222 PPC Amiga, or a kick-ass Intel i7 beast that chew¬†virtual machines for breakfast. If your computer can handle a modern website, then you can learn object pascal and work directly in the cloud.

desktop_embedded

The Smart Desktop running on cheap embedded hardware. The results are fantastic and the financial savings of using Smart Pascal on the kiosk client is $400 per unit in this case

Heck you can work off a $60 ODroid XU4, it has more than enough horsepower to drive the latest chrome or Firefox engines. All the compilation takes place on the server anyways. And if you want a Delphi vessel rather than phonegap (so that it’s a Delphi application that opens up a web-view in full-screen and expose features to your smart code) then you will be happy to know that this is being investigated.

More targets

There are a lot of systems out there in the world, some of which did not exist just a couple of years ago. FriendOS is a cloud based operating system we really want to support, so we are eager to get cracking on their SDK when that comes out. Being able to target FriendOS from Smart is valuable, because some of the stuff you can do in SMS with just a bit of code – would take weeks to hand write in JavaScript. So you get a productive edge unlike anything else – which is good to have when a new market opens.

As far as Delphi is concerned there are smaller systems that Embarcadero may not be interested in, for example the many embedded systems that have come out lately. If Embarcadero tried to target them all – it would be a never-ending cat and mouse game. It seems like almost every month there is a new board on the market. So I fully understand why Embarcadero sticks to the most established vendors.

ov-4f-img

Smart technology allows you to cover all your bases regardless of device

But for you, the programmer, these smaller boards can repsent thousands of dollars worth of saving. Perhaps you are building a kiosk system and need to have a good-looking user interface that is not carved in stone, touch capabilities, low-latency full-duplex communication with a server; not much you can do about that if Delphi doesnt target it. And Delphi is a work horse so it demands a lot more cpu than a low-budget ARM SoC can deliver. But web-tech can thrive in these low-end environments. So again we see that Smart can compliment and be a valuable addition to Delphi. It helps you as a Delphi developer to act on opportunities that would otherwise pass you by.

So in the above scenario you can double down. You can use Smart for the user-interface on a low power, low-cost SoC (system on a chip) kiosk — and Delphi on the server.

It all depends on what you are interfacing with. If you have a full Delphi backend (which I presume you have) then writing the interface server in Delphi obviously makes more sense.

If you don’t have any back-end then, depending on your needs or future plans, it could be wise to investigate if node.js is right for you. If it’s not – go with what you know. You can make use of Smart’s capabilities either way to deliver cost-effective, good-looking device front-ends of mobile apps. Its valuable tool in your Delphi toolbox.

Better infrastructure and rooting

So far our support for various systems has been in the form of APIs or “wrapper units”. This is good if you are a low-level coder like myself, but if you are coming directly from Delphi without any background in web technology – you wouldn’t even know where to start.

So starting with the next IDE update each platform we support will not just have low-level wrapper units, but project types and units written and adapted by human beings. This means extra work for us Рbut that is the way it has to be.

As of writing the following projects can be created:

  • HTML5 mobile applications
  • HTML5 mobile console applications
  • Node.js console applications
  • node.js server applications
  • node.js service applications (requires PM2)
  • Web worker project (deprecated, web-workers can now be created anywhere)

We also have support for the following operating systems:

  • Chrome OS
  • Mozilla OS
  • Samsung Tizen OS

The following API’s have shipped with Smart since version 1.2:

  • Khronos browser extensions
  • Firefox¬†spesific API
  • NodeWebkit
  • Phonegap
    • Phonegap provides native access to roughly 9 operating systems. It is however cumbersome to work with and beta-test if you are unfamiliar with the “tools of the trade” in the JavaScript world.
  • Whatwg
  • WAC Apis

Future goals

The first thing we need to do is to update and re-generate ALL header files (or pascal units that interface with the JavaScript libraries) and make what we already have polished, available, documented and ready for enterprise level use.

kiosk-systems

Why pay $400 to power your kiosk when $99 and Smart can do a better job?

Secondly, project types must be established where they make sense. Not all frameworks are suitable for full project isolation, but act more like utility libraries (like jQuery or similar training-wheels). And as much as possible of the RTL made platform independent and organized following our namespace scheme.

But there are also other operating systems we want to support:

  • Norwegian made Friend OS, which is a business oriented cloud desktop
  • Node.js OS is very exciting
  • LG WebOS,¬†and their Enyo application framework
  • Asustor DLM web operating system is also a highly attractive system to support
  • OpenNAS has a very powerful JavaScript application framework
  • Segate Nas OS 4 likewise use JavaScript for visual, universal applications
  • Microsoft Universal¬†Platform allows you to create truly portable, native speed JavaScript applications
  • QNap QTS web operating system [now at version 4.2]

All of these are separate from our own NAS and embedded device system: Smart Desktop, which uses node.js as a backend and will run on anything as long as node and a modern browser is present.

Final words

I hope you guys have enjoyed my little trip down memory lane, and also the plans we have for the future. Personally I am super excited about moving the IDE to the cloud and making Smart available 24/7 globally – so that everyone can use it to design, implement and build software for the future right now.

Smart Pascal Builder (or whatever nickname we give it) is probably the first of its kind in the world. There are a ton of “write code on the web” pages out there, but so far there is not a single hard-core development studio like I have in mind here.

So hold on, because the future is just around the corner ūüėČ

Smart Pascal: Information to alpha testers

April 3, 2017 Leave a comment

Note: This is re-posted here since we are experiencing networking problems at http://www.smartmobilestudio.com. The information should show up there shortly.

Our next-gen RTL will shortly be sent to people who have shown interest in testing our new RTL. We will finish the RTL in 3 stages of alpha before we hit beta (including code freeze, just fixes to existing code) and then release. After that we move on to the IDE to bring that up to date as well.

Important changes

You will notice that visual controls now have a ton of new methods, but one very interesting in particular called: ObjectReady. This method holds an important and central role in the new architecture.

You may remember that sometimes you had to use Handle.ReadyExecute() in the previous RTL? Often to synchronize an activity, setting properties or just calling ReSize() when the control was ready and available in the DOM.

To make a long story short, ObjectReady() is called when a control has been constructed in full and the element is ready to be used. This also includes the ready-state of child elements. So if you create 10 child controls, ObjectReady will only be called once those 10 children have finished constructing – and the handle(s) have safely been injected into the DOM.

To better understand why this is required, remember that JavaScript is asynchronous. So even though the constructor finish – child objects can still be busy “building” in the background.

If you look in SmartCL.Components.pas, notice that when the constructor finishes –it does a asynchronous call to a procedure named ReadySync(). This is a very important change from the previous version – because now synchronization can be trusted. I have also added a limit to how long ReadySync() can wait.¬†So if it takes to long the ReadySync() method will simply exit.

ObjectReady

As you probably have guessed, when ReadySync() is done and all child elements have been successfully created – it called the ObjectReady() method.

This makes things so much easier to work with. In many ways it resembles¬†Delphi and Freepascal’s AfterConstruction() method.

To summarize the call-chain (or timeline) all visual controls follow as they are constructed:

  • Constructor Create()
  • InitializeObject()
  • ReadySync
  • ObjectReady()
  • Invalidate
  • Resize

If you are pondering what on earth Invalidate() is doing in a framework based on HTML elements: it calls Resize() via the RequestAnimationFrame API. TW3GraphicControl visual controls that are actually drawn, much like native VCL or LCL components are – naturally invalidate the graphics in this method. But ordinary controls just does a synchronized resize (and re-layout) of the content.

When implementing your own visual controls (inheriting from TW3CustomControl), the basic procedures you would have to override and implement are:

  • InitializeObject;
  • FinalizeObject;
  • ObjectReady;
  • Resize;

Naturally, if you dont create any child controls or data of any type – then you can omit InitializeObject and FinalizeObject; these act as constructor and destructor in our framework. So in the JVL (Javascript Visual component Library) you dont override the constructor and destructor directly unless it is extremely important.

Where to do what

What the JVL does is to ensure a fixed set of behavioral traits in a linear fashion- inside an environment where anything goes. In order to achieve that the call chain (as explained above) must be predictable and rock solid.

Here is a quick cheat sheet over what to do and where:

  • You create child instances and set variables¬†in InitializeObject()
  • You set values and access the child instances for the first time in ObjectReady()
  • You release any child instances and data in FinalizeObject()
  • You enable/disable behavior in CreationFlags()
  • You position and place child controls in Resize()
  • Calling Invalidate() refreshes the layout or graphics, depending on what type of control you are working with.

What does a custom control look like ?

Its actually quite simple. Now I have included a ton of units here in order to describe what they contain, then you can remove those you wont need. The reason we have fragmented the code like this (for example System.Reader, System.Stream.Reader and so on) is because node.js, Arduino, Raspberry PI, Phonegap, NodeWebKit are all platforms that run JavaScript in one form or another – and each have code that is not 1:1 compatible with the next.

Universal code, or code that executes the same on all platforms is isolated in the System namespace. All files prefixed with “System.” are universal and can be used everywhere, regardless of project type, target or platform.

When it comes to the reader / writer classes, it’s not just streams. We also have binary buffers (yes, you actually have allocmem() etc. in our RTL) – but you also have special readers that work with database blobs, Bson attachments .. hence we had no option but to fragment the units. At least it makes for smaller code ūüôā

unit MyOwnControlExample;

interface

uses
  // ## The System namespace is platform independent
  System.Widget,           // TW3Component
  System.Types,            // General types
  System.Types.Convert,    // Binary access to types
  System.Types.Graphics,   // Graphic types (TRect, TPoint etc)
  System.Colors,           // TColor constants + tools
  System.Time,             // TW3Dispatch + time methods

  // Binary data and streams
  System.Streams,
  System.Reader, System.Stream.Reader,
  System.Writer, System.Stream.Writer,

  // Binary data and allocmem, freemem, move, copy etc
  System.Memory,
  System.Memory.Allocation,
  System.Memory.Buffer,

  // ## The SmartCL namespace works only with the DOM
  SmartCL.System,         // Fundamental methods and classes
  SmartCL.Time,           // Adds requestAnimationFrame API to TW3Dispatch
  SmartCL.Graphics,       // Offscreen pixmap, canvas etc.
  SmartCL.Components,     // Classes for visual controls
  SmartCL.Effects,        // Adds about 50 fx prefixed CSS3 GPU effect methods
  SmartCL.Fonts,          // Font and typeface control
  SmartCL.Borders,        // Classes that control the border of a control
  SmartCL.CSS.Classes,    // Classes for self.css management
  SmartCL.CSS.StyleSheet, // Create stylesheets or add styles by code

  { Typical child controls
  SmartCL.Controls.Image,
  SmartCL.Controls.Label,
  SmartCL.Controls.Panel,
  SmartCL.Controls.Button,
  SMartCL.Controls.Scrollbar,
  SMartCL.Controls.ToggleSwitch,
  SmartCL.Controls.Toolbar }
  ;

type

  TMyVisualControl = class(TW3CustomControl)
  protected
    procedure InitializeObject; override;
    procedure FinalizeObject; override;
    procedure ObjectReady; override;
    procedure Resize; override;
  end;

implementation

procedure TMyVisualControl.InitializeObject;
begin
  inherited;
  // create child instances here
end;

procedure TMyVisualControl.FinalizeObject;
begin
  // Release child instances here
  inherited;
end;

procedure TMyVisualControl.ObjectReady;
begin
  inherited;
  // interact with controls first time here
end;

procedure TMyVisualControl.Resize;
begin
  inherited;
  if not (csDestroying in ComponentState) then
  begin
    // Position child elements here
  end;
end;

CreateFlags? What is that?

Delphi’s VCL and Lazarus’s LCL have had support for CreateFlags for ages. It essentially allows you to set some very important properties when a control is created; properties that enable or disable how the control behaves.

  TW3CreationFlags = set of
    (
    cfIgnoreReadyState,     // Ignore waiting for readystate
    cfSupportAdjustment,    // Controls requires boxing adjustment (default!)
    cfReportChildAddition,  // Dont call ChildAdded() on element insertion
    cfReportChildRemoval,   // Dont call ChildRemoved() on element removal
    cfReportMovement,       // Report movements? Managed call to Moved()
    cfReportResize,         // Report resize? Manages call to Resize()
    cfAllowSelection,       // Allow for text selection
    cfKeyCapture            // flag to issue key and focus events
    );

As you can see these properties are quite fundamental, but there are times when you want to alter the default behavior radically to make something work. Child elements that you position and size directly doesn’t need cfReportReSize for instance. This might not mean much if its 100 or 200 child elements. But if its 4000 rows in a DB grid, then dropping that event check has a huge impact.

ComponentState

Yes, finally all visual (and non visual) have componentstate support. This makes your code much more elegant, and also way more compatible with Delphi and Lazarus. Here are the component-states we support right now:

  TComponentState = set of (
    csCreating,   // Set by the RTL when a control is created
    csLoading,    // Set by the RTL when a control is loading resources
    csReady,      // Set by the RTL when the control is ready for use
    csSized,      // Set this when a resize call is required
    csMoved,      // Set this when a control has moved
    csDestroying  // Set by the RTL when a control is being destroyed
    );

So now you can do things like:

procedure TMyControl.StartAnimation;
begin
  // Can we do this yet?
  if (csReady in ComponentState) then
  begin
    // Ok do magical code here
  end else
  //If not, call back in 100ms - unless the control is being destroyed
  if not (csDestroying in ComponentState) then
    TW3Dispatch.Execute(StartAnimation, 100);
end;

Also notice TW3Dispatch. You will find this in System.Time and SmartCL.Time (it is a partial class and can be expanded in different units); we have isolated timers, repeats and delayed dispatching in one place.

SmartCL.Time adds RequestAnimationFrame() and CancelAnimationFrame() access, which is an absolute must for synchronized graphics and resize.

Other changes

In this post I have tried to give you an overview of immediate changes. Changes that will hit you the moment you fire up Smart with the new RTL. It is very important that you change your existing code to make use of the ObjectReady() method in particular. Try to give the child elements some air between creation and first use – you get a much more consistent result on all browsers.

The total list of changes is almost to big to post on a blog but I did publish a part of it earlier. But not even that list contains the full extent. I have tried to give you an understanding

Click here to look at the full change log

Smart Pascal: Busting browser storage limits

April 2, 2017 Leave a comment

Sessionstorage is the name for a browser’s in-memory only storage. Meaning that it’s essentially a ram-disk that is just deleted when you navigate away from the website or close the browser.

Sessionstorage has also been deprecated, so you should avoid using it and go for Localstorage, or just use a raw, untyped uint8 array instead.

Or should you?

Ensuring 64 megabytes minimum

Browsers do not behave identically across devices. Try to get a concurrent reading of something as simple as drawing sprites, and you will quickly notice that even the same device families (Android, iOS and Microsoft) can behave differently between versions – and even builds (revisions).

On embedded systems or thin clients with very little memory, allocating large chunks ot uint8 arrays is not going to work. One of my test thin-client machines has only 512 megabyte ram – and it would throw an exception if I tried to allocate more than 20 megabyte of continuous memory (again, as an array of uint8 bytes).

Using the dark side of the force

Screenshot

Offline means the system boots from a local cache disk

While testing Smart code on this little device, I noticed that quite large images loaded just fine. So where I was not allowed to allocate more than 20 megabytes, the browser would happily load in pictures taking up over 50 megabyte of pixel data?

It then struck me that the maximum limit of a picture, which is enforced by the DIB Api (at least on Windows desktop and embedded), is 4000 x 4000 pixels. Since each pixel is 32 bits (4 bytes, RGBA) that my friend is 64 megabytes right there!

I created a new class that inherits from the virtual-filesystem that Smart Pascal uses, created an off-screen image object in the constructor – and then made a simple but effective “bytes to scan line” calculation routine. So whenever the need for more data grew, it would¬†first grow the picture so it could hold the data (and shrink it again) on demand.

Humble but meaningful

Now 64 megabytes might not seem like much in our day and age, but if you are on holiday and want to connect to your home NAS – 64 megabytes of available ram makes a huge difference. Remember that localstorage only allows between 5 and 10 megabytes.

I should mention that using an image as a buffer makes little sense on a full Windows PC, a Mac or a Linux box. These system will page memory to disk and you will most likely never encounter the 20 megabyte barrier I experienced on this low-end Dell thin client device. But considering that hotels, motels and b&b often have thin clients setup for their customers (read: you) – The Smart desktop has to take height for it.

 

 

 

 

 

Smart Pascal: Update specifics

January 20, 2017 Leave a comment

With what is probably the biggest update Smart Mobile Studio has seen to date only a couple of weeks away, what should the Smart Pascal programmer know about the new RTL?

I have written quite a few articles on the changes, so you may want to backtrack and look at them first – but in this post I will try to summerize two of the most important changes.

Unit files, a background

The first version of Smart Mobile Studio (sms) was designed purely for webkit based, HTML5 applications. As the name implies the initial release was designed purely for mobile application development; with the initial target firmly fixed on IPhone and IPad. As a result, getting rid of webkit exclusive code has been very time-consuming to say the least.

At the same time we added support for embedded devices, like the Espruino micro-controller, and in leu of that process -it became clear that the number of devices we can support is huge. As of writing we have code for more than 40 embedded devices lined up, ready to be added to the RTL (after the new RTL is in circulation).

Add to this the fact that node.js is now the most powerful cloud technology in the world, the old “DOM only”, webkit oriented codebase was hopelessly narrow minded and incapable of doing what we wanted.¬†Something had to give, otherwise this would be impossible to maintain – not to mention document in any reasonable logical form.

A better unit system

I had to come up with a system that preserved universal code – yet exposed special features depending on the target you compile for.

So when you compile for HTML5 you get all the cool stuff the document object model (DOM) has to offer, but also full access to the universal units. Same goes for node.js, its has all the standard stuff that is universal – but expose a ton of JS objects that exist nowhere else.

What I ended up with was that each namespace should have the same units, but adapted for the platform it represents. So you will have units like System.Time.pas, which represents universal functionality regarding timing and synchronization; But you are supposed to use the one prefixed by your target namespace. For instance, SmartCL.Time or SmartNJ.Time. In cases where the platform doesnt expose anything beyond the universal code, you simply fall back on the System.*.pas unit.

namespace_scheme

It may look complex, but it’s super easy once you get it

The rules are super simple:

  • Coding for the browser? Use the units prefixed with SmartCL.*
  • Coding for node.js? Use the units prefixed with SmartNJ.*
  • Coding for embedded? Use the units prefixed with SmartIOT.*

Remember that you should always include the system version of the unit as well. So if you include SmartCL.Time.pas, add System.Time.pas to the uses list as well. It makes it easier to navigate should you want to CTRL + Click on a symbol.

Better synchronization engine

JavaScript is async by nature. Getting JavaScript to behave like traditional object pascal is not only hard, it is futile. Besides, you dont want to be treated like a child and shielded from the reality of JavaScript – because that’s where all the new cool stuff is at.

In Delphi and Lazarus we are used to our code being blocking. So if you create child objects in your constructur, these objects are available there and then. Well, thats not how JavaScript works. You can create an object, the code continues, but in reality the object is not finished yet. It may in fact be assembling in the background (!)

To make writing custom-controls sane and human, we had to change a few things. So in your constructor (which is InitializeObject in our RTL, not the actual class constructor) you are expected to create child object – but you should never use them.

Instead there is a method called ObjectReady() which naturally fires when all your child elements have been constructed properly and the control has been safely injected into the document object model. This is where you set properties and start to manipulate your child objects.

synchro

So finally the setup process is stable, fast and working as you expect. As long as you follow this simple rule, your controls will appear and function exactly like you expect them to.

  • Never override Create and Destroy, use InitializeObject and FinalizeObject
  • Override ObjectReady() to set initial values

Creation flags

We have also added creation flags to visual controls. This is a notion familiar to both Delphi’s VCl and Lazarus LCL component structures. It allows you to define a few flags that determines behavior.

For instance, if your control is sized manually and will never really need to adapt – then it’s a waste of time to have Resize() fire when the size is altered. Your code will run much faster if the RTL can just ignore these changes. The flags you can set are:

  TW3CreationFlags = set of
    (
    cfIgnoreReadyState,     // Ignore waiting for readystate
    cfSupportAdjustment,    // Controls requires boxing adjustment (default!)
    cfReportChildAddition,  // Dont call ChildAdded() on element insertion
    cfReportChildRemoval,   // Dont call ChildRemoved() on element removal
    cfReportMovement,       // Report movements? Managed call to Moved()
    cfReportResize,         // Report resize? Manages call to Resize()
    cfAllowSelection,       // Allow for text selection
    cfKeyCapture            // assign tabindex, causing the control to issue key and focus events
    );

To alter the default behavior you override the method CreationFlags(), which is a class function:

class function TW3TagObj.CreationFlags: TW3CreationFlags;
begin
  result := [
    // cfAllowSelection,
    cfSupportAdjustment,
    cfReportChildAddition,
    cfReportChildRemoval,
    cfReportMovement,
    cfReportReSize
    ];
end;

Notice the AlloSelection flag? This determines if mouse selection of content is allowed. If you are making a text display, or editor, or anything that the user should be allowed to select – then you want this flag set.

The cfKeyCapture is also a flag many people have asked about. It determines if keyboard input should be allowed for the element. By default this is set to OFF. But now you dont have to manually mess around with element attributes. Just add that flag and your control will fire the OnKeyPress event whenever it has focus and the user press a key.

Tweening and fallback effects

In the last update a bug managed to sneak into our SmartCL.effect.pas library, which means a lot of people havent been able to fully use all the effects. This bug has now been fixed, but we have also added a highly efficient tweening library. The concept is that effects will be implemented both as hardware acellerated, GPU powered effects – but also with CPU based tweening alternatives.

For mobile devices it is highly adviced to use the GPU versions (the normal SmartCl.Effects.pas versions).

When you add the unit SmartCL.Effects.pas to your forms, what happens is that ALL TW3MovableObject based controls are suddenly extended with around 20 fx prefixed methods. This is because we use partial classes (which means classes can be extended).

This is super powerful, and with a single line of code you can make elements fly over the screen, shrink, zoom out or scale in size. You even get a callback when the effect has finished – and you can daisy-chain calls to execute a whole list of effects in sequence.

FMyButton.fxSizeTo(10, 10, 200, 200, 0.8, procedure ()
  begin
   writeln("Effect is done!");
  end);

And execute in sequence. The effects will wait their turn and execute one by one.

FMyButton.fxMoveTo(10,10, 0.8)
  .fxSizeTo(100,100, 0.8)
  .fxFadeTo(0.3, 0.8);

Stay tuned for more info!

Smart Pascal: Node.js by example

January 16, 2017 Leave a comment

nodeThe update for the Smart Mobile Studio RTL is nearing completion. We still have a little to go, but all in all we have cleaned up the VJL considerably and made a much better organization. We now have 3 namespaces (read: groups of units), namely: System, SmartCL and SmartNJ. The latter representing the Node.js aspect of our codebase. We also have a fourth namespace in the making, namely Embedded. Currently our embedded support is limited to Espruino – but I have begun working on the Arduino and Arduino mega codebase. When these are complete they will all be in the Embedded namespace.

Ok, node.js so where do I begin. Perhaps its best to explain a few fundamental keys to how the new RTL is organized so you understand just much Smart gives you over vanilla JavaScript.

I mentioned the system namespace above, but what exactly does that mean? The system namespace contains universal code. Code that runs on all JavaScript platforms regardless of a DOM being present or not. So you get to enjoy stuff like:

  • Traditional TStream, TMemoryStream and TFileStream
  • Codec classes
  • Memory allocation and blue pointers
  • Encryption classes
  • Delayed execution (TW3Dispatch, system.time.pas)
  • Event objects
  • Filesystem
  • Datatype conversion (system.type.convert)
  • .. and the whole system namespace

Notice the TFileStream there? You will find that the system namspace contains an abstract implementation of a class called TFileSystem. And this is where things get funky. When you create a SmartCL application (that is, a visual HTML5 application), the implemented filesystem is in SmartCL.Filesystem. When you create¬†a node.js application, the implemented filesystem can be found in SmartNJ.filesystem. See what I’m getting at here?¬†The idea is that regardless of the runtime environment, you will always be able to use the same code no matter what engine you run on.

Ok, hopefully that gave you some context to work with, now let’s move on and create a fancy server for node.js !

Sexy server time

We are going to create a simple yet powerful UDP client / server application. I picked this because it’s sort of the odd-ball in our collection of server objects. UDP is connection-less and is ridiculously easy to use. At the same time there is no verification and assurance of delivery (that’s the flipside). But UDP is excellent for “live” logging, or for inter-service communication between micro-services on the same network or router.

Right, let’s just jump straight in:

var Server: TNJUDPServer;
Server := TNJUDPServer.Create;

Server.OnMessage := procedure (Sender: TObject; Data: variant;
  RequestInfo: TNJUDPRequestInfo)
begin
  writeln("Recieved data:" + string(data));
end;

Server.OnError := procedure (Sender: TObject;
  ErrorObj: TJsErrorObject)
begin
  writelnF("Error[%s]: %s", [variant(ErrorObj.stack).toString(), ErrorObj.message]);
end;

Server.OnClose := procedure (Sender: TObject; ErrorObj: TJsErrorObject)
begin
  writeln("Server closed");
end;

Server.OnAfterServerStarted := procedure (sender: TObject)
begin
  server.Send("Your first Smart server is now running");
end;

Server.Port := 1881;
Server.Address := '127.0.0.1';
Server.MulticastLoopback := true;
Server.Broadcast := true;
server.Exclusive:= false;
Server.Start();

Congratulations! You have just created your first node.js powered UDP server! It doesnt get much easier than this does it? Try to guess how much JS code you would have to write to get all the perks of the system namespace + the node.js namespace.

Some guy commented on an article I wrote the other day, asking me to give him a single reason why Smart Mobile Studio is better than just writing JavaScript. The answere is simple: producivity. Some of the code you get in a simple class in Smart represents hundreds or thousands of lines of JavaScript code.

What do you think is more productive? Maintaining 2 megabyte of raw JavaScript code; a binary kebab of spaghetti code that can go belly up if you missplace as much as a comma. Or the 2k of clean, easy to read, easy to understand object oriented pascal code?

I rest my case.

Right, with the server done already, let’s have a peek at the client:

LClient := TNJUDPClient.Create;
LClient.Port := 1881;
LClient.Address := "localhost";
LClient.Bindings.Add('127.0.0.1',1881);
LClient.OnAfterStarted := procedure (sender: TObject)
begin
  LClient.Send("Client is now active", 1881, 'localhost',
  procedure (ErrorObj:   TJsErrorObject)
  begin
    writeln("An error occured:" + ErrorObj.Stack.ToString());
  end);
LClient.Active := true;

And that my friend is pretty much it!

Node.js really is awesome once you get to approach it on our terms. When you come to the JavaScript virtual machine through the OOP layer that Smart Pascal gives you, it’s a whole different reality.

Write a cluster ready, platform independent client/server application in less time than it takes to boot up Visual studio

Write a cluster ready, platform independent client/server application in less time than it takes to boot up Visual studio

What is the benefit here you might ask? All of this can be done in Delphi and Lazarus without much problem. The benefit is this: the generated code is platform independent, it will run on any platform as long as node.js is installed. It will behave identical regardless of operating-system. And you can cluster, clone and replicate instances to your heart’s desire.

Node.js is also easy to host, cheap and accessible. Native hosting is expensive and requires much more work. When something goes wrong inside a 200 megabyte service hosted on Amazon.. I have been there. Having to track down the bugs, re-build the executable while your boss is going mental – spend 10 minutes just uploading the damn thing, then you have to uninstall the service, reboot the whole instance, re-install the new .exe file and hope that you did manage to catch that bug.

Compare that to fixing the bug, uploading the new JS file and then inform PM2 that you need to hot-swap the service code. It’s a whole new paradigm.

Websocket server

With UDP behind us, let’s look at something a lot more complex. And when I write complex, I dont mean for you. Websocket is the fastest growing protocol standard these days. It’s basically an extension of the ordinary HTTP protocol. It is designed for long-term connections (read: not stateless, single shot operations like http) and is full duplex and async. So the server can talk to the client and visa versa without having to wait it’s turn. Just fire off as many messages as you like, whenever you like – and it will arive in the same order on the other side.

var
LServer: TNJWebSocketServer;

LServer := TNJWebSocketServer.Create;
LServer.ClientTracking := true;
LServer.Port := 1881;

// Setup our own protocol commands
LServer.On("command1", procedure (const Socket: JWsSocket; const Data: variant)
  begin
    writeln("Command #1 executed, recieved data:");
    writeln(data);
  end);

LServer.On("command2", procedure (const Socket: JWsSocket; const Data: variant)
  begin
    writeln("Command #2 executed");
    // broadcast a "hello" response with a byte array to *ALL*
    // connected clients. Making chat applications is very easy here
    socket.Emit("Hello", [12,13,14,14]);
  end);

LServer.Active := true;

If you look closely you will notice the use of the On keyword. This is what we use to define our server-side protocol. It’s essentially trigger words that you associate with a piece of code. When the server recieves a message it will read the message-name, look up a associated code and execute it. But let’s look at the background for this so you dont end up downloading the wrong package.

Normally you install the node.js websocket package first, and then you install something called socket.io on top of that. The On keyword we used above is actually not a part of the default websocket standard (which is more low level). This functionality is provided by a separate package called socket.io.

But, since most people install and use websocket just to use socket.io, it makes sense to merge these two packages into a single, stand-alone distro that gives you everything in one go. And this package is cleverly named Websocket-IO (it’s just knockout names isnt it).

I usually install node packages in my project output folder

I usually install node packages in my project output folder

If you want to read more about websocket-io, head over to the npm website and have a peek at: https://www.npmjs.com/package/nodejs-websocket

Client: Needless to say, the client class is just as simple so I dont really see a point in repeating that. In fact, all servers and clients inherit from the same base-class.

What should be stressed is that websocket is easy to use from Delphi as well. So if you have a native client and want to talk with your node.js server, I strongly suggest you stay clear of REST (which is one of the most wasteful protocols ever invented) and instead use websocket.

Note: since websocket is an extension to http, it is perfectly safe to use in a commercial environment. REST requires a lot more CPU and generates much more data on the network than websocket. The most time consuming and intensive aspects of a http call is during connection, and websocket is based on async and full duplex communication, with message caching and more handled automatically – so there is no reason why you should chose REST over websocket.

Websocket allows you to broadcast messages, or to filter messages based on criteria. Messaging and data-flow protocols are very easy to implement

Websocket allows you to broadcast messages, or to filter messages based on criteria. Messaging and data-flow protocols are very easy to implement

When you add the fact that websocket is also allowed from the browser, you can offer your services to the Javascript community without any extra development. So you get to cater for both native and JS clients. Being able to connect and interact with your custom servers directly from the browser opens up new possebilities. Want to display live information? With a full duplex connection the server can now inform you whenever something has changed instead of your clients polling on interval.

A Delphi extension to websocket (for Indy) was implemented and released by Andre Mussche a while back. The code is super easy to work with and bridges the world of Smart Pascal and Delphi quite nicely:
https://github.com/andremussche/DelphiWebsockets

If you need information about how to use websocket regardless of language, just google the topic. There are thousands of resources out there.

Last but not least

I hope you have found this super simple introduction to node.js server coding inspirational. Node.js is a huge topic and there are roughly 350.000 different packages available online for it. Writing wrapper classes for node packages is not hard either. You do have to know your way around object pascal and Javascript, but if you look at my units and how we have solved it, you should pick it up very fast.

In the future I hope to generate a 1:1 import tool that will download, convert and install packages for you automatically.

Cheers guys!

For those about to scroll, we salute you

June 10, 2016 1 comment

Came up with a better solution to the scrolling problem for Smart Mobile Studio. While I hate having to write code for Internet Explorer, it is nice to have a system that works everywhere. But that means more abstraction and “drivers” type classes.

The scrolling model

TW3Scrollbox and it’s variation(s) implement different types of scrolling. TW3Scrollbox has a very fast non-momentum scroll, making it perfect for displaying detailed information. But it would be nice if we could choose right?

The model I have come up with is super simple, especially in the upcoming RTL where we finally have non-visual components. I give you – TW3ScrollController

scroll_model

In the present model, TW3Scrollbox deals with scrolling directly. Actually the scrolling is implemented in the TW3ScrollContent control, mapping touch and mouse events directly. This turned out to be a remarkably fast way of moving things around. I really did not expect IE to keep up, but it’s perfectly pristine!

In the momentum-scroll example I posted, control of scrolling is handled by the container rather than content. This is very fast on webkit, and I also tried it on Microsoft mobile and Windows 10 mobile – and it’s very fast there. But for some reason the desktop Internet Explorer is slow and the content jitters a bit.

The culprit is not my¬†code or approach, it’s actually something else. Internet Explorer and Edge are the only browsers that implements OnReSize() events. No other browser has this. In the SMS RTL we have to manually figure out when to call ReSize (based on the BeginUpdate, AddControlState, EndUpdate methods).

In the momentum scroller I use SetBounds() to keep the content within the horizontal bounds of the control. This causes an extra call to Resize every time the content moves even a pixel (even though it shouldnt, because the size doesnt change). So yeah, fixing that will make all the difference. I’m going to nail this thing once and for all, just like I did with font measurements way back.

TW3ScrollController

But isolating the code that actually deals with scrolling in separate, non visual components that you can attach to the TW3ScrollBox makes sense. Rather than hardcoding everything into a huge, spaghetti monster unit — we can now isolate different scroll methods in their own units (keeping those bytes down).
Normal per-pixel scrolling, momentum scrolling, CSS3 animation based scrolling, tween based (cpu) scrolling. It gives us some options – and allows you to implement your own variations if you find mine lacking.

I’m also giving the browser driver a much deserved overhaul. Getting the browser type and version info should be easy (and humanly readable). And since you may want to pick different scroll methods depending on the browser type — being able to check if your running on Edge for the desktop, or IE on a mobile device… well, it should be there. End of story.

iScroll

I really want iScroll to be the standard scroll library for Smart Mobile Studio, but since people feel it’s hard to use and adapt to — I may end up doing the unthinkable and re-write it in object pascal from scratch. But iScroll5 really is so much better. It has been developed and tested on a plethora of devices for six years now.

It even does things the built-in browser scrolling (for the browsers that allows this, yeah im looking at you Safari!) doesn’t deal with.

But I have enough on my plate right now, so iScroll porting will have to wait.

Selecting text, Smart Mobile Studio

May 31, 2016 Leave a comment

A really good question came my way regarding text selection and why ordinary browser behavior is disabled in Smart Mobile Studio applications. It was Jarto Tarpio that posted it on the SMS Facebook page, and I was a little surpriced at first to say the least.

Now the VJL was designed to look and feel more or less identical to any ordinary, native UI framework. This means that ordinary HTML text selection (marking text with your mouse) is something you dont want to have.

But what Jarto pointed out was that there is something fishy about how it’s implemented in Smart at the moment, because even when he removed the CSS rules that disables this, selecting text is still not possible.

That one line ..

Two RTL updates ago I remember dealing with this exact topic. I was happy to find that CSS had a couple of universal rules you could use, which meant that i could remove the code I was using to avoid selection ever taking place.

Sadly, I must have forgotten all about it (or it got accidentally filtered out during a manual unit merge) because it was till there (!) In other words not only did the CSS make sure you couldnt enable text selection, the startup code for all TW3TagObj derived classes kidnaps the OnSelectStart event — effectively killing initializing a selection at all.

Patching the RTL yourself

The change you need is basically a “one liner”, and it wont affect your programs at all. If you open up SmartCL.Components (right click the unit in your units clause), hit ALT + F which brings up the search dialog and then enter “TW3CustomControl.HookEvents” that’s where the problem lies:

procedure TW3CustomControl.HookEvents;
begin
  inherited;
  Handle['onclick'] := @CBClick;
  //w3_bind2(Handle, 'onselectstart', CBNoBehavior); //Remove this line !!
  w3_bind2(Handle, 'onfocus', CBFocused);
  w3_bind2(Handle, 'onblur', CBLostFocus);
end;

So simply delete the first “w3_bind2” call, the one that assigns OnSelectStart to the “no operation” event handler CBNoBehavior — now go to your main form unit, type something and then click Save.

Note: The editor doesnt monitor RTL files for edit-changes since these units are not meant to be edited. So you have to alter something in your project and force the IDE to allow your change to be saved.

Making selection an option

With that nasty (and no longer needed) line of code out of the way – I have added two new methods to the VJL that gives you 100% control over content selection. This will be in the next update but for those that cant wait, the following methods have been added to TW3MovableControl:

function GetContentSelectionMode: TW3ContentSelectionMode;virtual;
procedure SetContentSelectionMode(const NewMode: TW3ContentSelectionMode);virtual;

Here is the code if you want to get these features straight away:


type
  TW3ContentSelectionMode = (
    tsmNone,
    tsmAuto,
    tsmText,
    tsmAll,
    tsmElement
    );

function TW3MovableControl.GetContentSelectionMode: TW3ContentSelectionMode;
begin
  var CurrentMode := Handle.style[BrowserAPI.PrefixDef("user-select")];
  if (CurrentMode) then
  begin
    case TVariant.AsString(CurrentMode).ToLower() of
    'auto': result := tsmAuto;
    'text': result := tsmText;
    'all': result := tsmAll;
    'element': result := tsmElement;
    else
      result := tsmNone;
    end;
  end else
  begin
    Handle.style[w3_CSSPrefixDef("user-select")] := 'none';
    result := tsmNone;
  end;
end;

procedure TW3MovableControl.SetContentSelectionMode
  (const NewMode: TW3ContentSelectionMode);
begin
  case NewMode of
  tsmAuto:  Handle.style[w3_CSSPrefixDef("user-select")] := 'auto';
  tsmAll:   Handle.style[w3_CSSPrefixDef("user-select")] := 'all';
  tsmText:  Handle.style[w3_CSSPrefixDef("user-select")] := 'text';
  tsmNone:  Handle.style[w3_CSSPrefixDef("user-select")] := 'none';
  tsmElement: Handle.style[w3_CSSPrefixDef("user-select")] := 'element';
  end;
end;

Writing your custom controls with selection turned on

Ok, first let’s write a class that allows text-selection. We are going to use a DIB, which is the default element of TW3CustomControl, so this will be a short example:

type

TSelectTestControl = class(TW3CustomControl)
protected
  procedure ObjectReady;override;
public
  property Text:string read (GetInnerText) write (SetInnerText(Value));
end;

procedure TSelectTestControl.ObjectReady;
begin
  inherited;
  SetContentSelectionMode(tsmText);
end;

Since user-select is set to “none” in all our CSS style themes, enabling editing means telling the DOM (document object model) that this control does allow text selection. In the above example I do that in the ObjectReady method, just to make sure the DIV TSelectTestControl represent is in the clear and has been created successfully and injected into the DOM.

So lets go back to our main form and create an instance of our new control. You should also drop a TW3DIVHtmlElement on the form first, so we have something to compare with.

Ok, here is the form unit completed:

unit Form1;

interface

uses
  System.Colors, SmartCL.Controls.Elements, SmartCL.System,
  SmartCL.Graphics, SmartCL.Components, SmartCL.Forms,
  SmartCL.Fonts, SmartCL.Borders, SmartCL.Application;

type

  (* our spanking new control. Not a TW3DIVHtmlElement in sight! *)
  TSelectTestControl = class(TW3CustomControl)
  protected
    procedure ObjectReady;override;
  public
    property Text:string read (GetInnerText) write (SetInnerText(Value));
  end;

  TForm1 = class(TW3Form)
  private
    {$I 'Form1:intf'}
  protected
    procedure InitializeForm; override;
    procedure InitializeObject; override;
    procedure Resize; override;
  end;

implementation

{ TForm1 }

uses system.types, system.time;

procedure TSelectTestControl.ObjectReady;
begin
  inherited;
  SetContentSelectionMode(tsmText);
end;

procedure TForm1.InitializeForm;
begin
  inherited;
  // this is a good place to initialize components
  var LTemp := TSelectTestControl.Create(self);
  LTemp.background.fromColor(clRed);
  LTemp.Text:="this is some text";
  LTemp.setBounds(10,10,200,200);
  W3DivHTMLElement1.innerhtml := "<i>This is some text";
end;

procedure TForm1.InitializeObject;
begin
  inherited;
  {$I 'Form1:impl'}
end;

procedure TForm1.Resize;
begin
  inherited;
end;

initialization
  Forms.RegisterForm({$I %FILE%}, TForm1);
end.

And the results are exactly what we wanted:

select text

The red box is our control, as you can see you can now mark the text

Final words

I know this has been a topic many people have asked about in the past. But this time we have made sure this is possible (and possible per individual control).

Well what are you waiting for! Go patch that RTL right now and get cracking ūüôā

 

Smart Pascal, what’s next?

April 11, 2016 3 comments

For those of you who have been following my developer’s log over the years, both here (and on www.smartmobilestudio.com), I have a tradition to write a few words about where we are heading next after each release. So with the release of version 2.2 here is a peek of the future and what we will be cooking up in the lab next. Just for you!

The secret lab that shall not be named

The secret lab where all the magic happens

As a team we have worked hard for the past years, actually I started on it back in 2010, Eric joined in early, somewhere between 2010 and 2011 – at the same time as J√łrn E. ¬†Angeltveit (owner and CEO of Optimale Systemer AS, the original publishing company). And together with solid developers like Primoz Gabrielcic, Christian Budde and Andre Mussche, we formed the OP4JS¬†consortium in 2012. Dedicated to a new object pascal for a new¬†age.

That was 4 years ago. And today Smart Pascal is not only a distinct dialect of object pascal, it is also a licensed, stock based company: The Smart Company AS. Standing on its own two feet and continuing to grow.

We have seen Smart Mobile Studio (SMS) go from being an “Adobe Flash” like programming environment, into a fully fledged IDE with a decent form designer, support for packages (yes you can create packages and register components just like Lazarus and Delphi).

But is that all it’s ever going to be? Hardly. And I know people have a lot of questions about where we are heading, so I figured it’s time to shed some light on the subject.

Platform independence

 

In our recent release we have given our customers a whole year worth of research and development. I know people expected more frequent updates, perhaps 4 updates which was our initial goal – but had we done so, the RTL would have seen incomplete for 3 of those, so we decided to wait until the foundation was solid and in place.

NodeJS designer uses TTreeView as explained

The NodeJS service editor will make its debug very soon

To demonstrate: take something simple; Let’s take TMemoryStream. Easy right? Its been with Delphi since day 1, its one of the first things you learn about in DIY programming books, and considering we live in 2016 streams is something you take for granted.

Well guess what, that could not be further from the truth when it comes to JavaScript.¬†The wonderful language itself may hold 51% of the global computing markedshare, but like the author himself has stated on numerous occasions: it was slapped together in record time and was nowhere near complete when Netscape pushed it. So streams, strong type checking, datatype conversion, sandboxed memory management and all the stuff we would expect a modern language to support — it just isn’t there. It has slowly appeared in various unfriendly forms (and Typescript naturally, which imitates aspects of Smart Pascal, cheeky Anders Hejlsberg), but to overcome these shortcomings we have had to implement even the most fundamental¬†functionality from scratch, in Smart Pascal¬†itself.

Something as trivial as converting a 32 bit integer into 4 bytes, or turning a TDateTime (64 bit float) into an 8 byte array is something JavaScript developers have struggled with for quite some time (using bit-shift is the typical solution, we use a faster method). They have been reduced to cheap hacks, like packing bytes as character data; the proverbial assault on the heap.

So before we could even begin to introduce stuff like TStream, we first had to research and implement consistent behavior between  platforms. What platform you may ask? Doesnt Smart Pascal target normal browsers? Absolutely, but there is a difference between browser, engine and renderer. And we also have much larger ambitions than just browser apps.

Browsers render, engines execute

Browsers may look the same but beneath the hood they can be as different as Delphi is from Pearl or Python. The list of browsers we want to support is long and growing, separated by builds (age of mobile device, type of device and even PC graphics card capabilities):

  • There is¬†Mozilla Firefox
  • Safari, extended version of webkit
  • Chrome, standard webkit
  • Internet Explorer
  • Microsoft Spartan browser
  • Opera
  • Epiphany
  • Midori,¬†customized webkit

And those are just for desktop PC’s or embedded systems (like the Raspberry PI) running Windows¬†, OS X or Linux in some form or another. Six of these browsers are further¬†separated by desktop and mobile editions¬†where differences can be¬†monumental (like CSS gradients suddenly altering direction and syntax between CEF2 and CEF3).

Technically speaking each browser and engine (where engine means the JavaScript execution module, like Webkit V8, Mozilla Spidermonkey or IO.JS) have both subtle and radical differences which we have to care about. Tiny but important details like timing issues, synchronization differences, priorities for dispatching and performance considerations.

Now throw NodeJS and IO.JS (initially a fork of NodeJS which has gone in a slightly new direction) into the mix; combine with embedded SoC running JavaScript natively (more and more hardware support JS as their primary automation language) and you get the idea of just how much our work is involved. Things like converting integers to bytes and back again seem trivial, child like even, but they are of utmost importance. Something as simple as using UInt8ClampedArray instead of UInt8Array on the wrong engine, browser or build can kill your application before it even starts.

So streams, just to get back to that, involves a little more than just slapping together a class. To get streams working we first had to build the foundation for streams to exist in the first place:

  • Replicate object pascal strict types
  • Research fastest and most efficient way of doing binary conversion
  • Implement native type conversion in JavaScript itself
  • Implement real memory handling:
    • The ability to allocate memory (TAllocation)
    • The ability to modify memory¬†(TBinaryData)
  • Differentiate between various encoding standards
  • Implement marshaled pointers (offset references)

Take a moment to reflect on how much research, testing, prototyping and solution finding these simple concepts represents. Things Delphi has had over 21 years to reach, evolve and perfect.

And before Delphi became the product we all use and love today, it first went through 4 year of evolution as “Turbo Pascal for Windows”, releasing a grand total of 4 editions before that amalgamated into Delphi. And before that, seven years of Turbo Pascal earning Borland millions of dollars.

Engine and platform standard library

With version 2.2 just out the door, you get to enjoy elegant solutions to things JavaScript developers struggle with to this very day. Streams, memory management, writing custom controls with full inheritance, composite layout — all with rich support for NodeJS and various mobile and embedded platforms. But we are not done, not by a long-shot.

Writing object pascal services using standard RTL classes, running on affordable NodeJS hosting servers Рfrom small vendors to giants like Microsoft Azure and Amazon cloud service is the next big goal.

NodeJS and IO.JS may look like plain old JavaScript but they are worlds apart from the environment you find in a browser. We are already hard at work separating the RTL into 3 distinct namespaces. This complete picture will appear later, most likely in version 2.2.5 or even 2.3. And with it you will enjoy many of the top-level benefits Delphi and Lazarus have in their native codebase. But this time, you can run it anywhere, even on your TV which no doubt have a browser!

  • Unified standard classes between namespaces
  • Node and IO Service application type
    • Command server
    • WebSocket server
    • DB server (and drivers)
  • Unified DB API
  • Non visual components
  • Assets management (pre-loading of resources for easy distribution)
  • TActions for visual elements
  • IDE Embedded development

Embedded development

This is being worked on by a third-party and will be exciting for IOT programmers using the Raspberry PI (for example). A version of ARM Ubuntu is being tailored to communicate with our IDE. The Linux image will run NodeJS services you write in Smart Mobile Studio and can also (as a bonus) render a visual front-end application directly to the framebuffer.

With the internet of things (IOT) you need a language which reaches all platforms

With the internet of things (IOT) you need a language that connects

In other words: you will have full control over the entire device, and can use SMS as the tool to create single-application IOT devices. Examples of products you can create with this technology are Apple TV or Plex devices, ChromeStick, NetFlix, kiosk systems; Development from A-Z will can be done directly inSmart Mobile Studio. The IDE will also communicate with the IOT device, uploading your compiled NodeJS services, databases, assets and visual project UI code.

The Smart Desktop runs on Raspberry PI on boot, with NodeJS dealing with system level operations in the background.

The Smart Desktop runs on Raspberry PI on boot, with NodeJS dealing with system level operations in the background. So much fun to work with!

While I love all things Smart Pascal, this is by far my favorite. IOT is important and allows you to create just about everything you can imagine.

And the beauty? It’s all done using standard protocols. Your NodeJS service can be hosted on Azure or Amazon, or just run quietly on your Raspberry PI or Beaglebone. It makes no difference. The code and classes will be exactly the same.

You are going to love it.

 

Smart wrappers, twirk like jQuery

November 4, 2015 3 comments

I was¬†asked how to better implement a “JQuery like syntax” – but for Smart Pascal rather than surrendering to alien and cryptic JavaScript.¬†Well, there are many ways to skin a cat, but below is a typical example of how can wrap interesting functionality more or less like jQuery does it; Except within the confines of Smart Pascal.

But naturally, object pascal is a much more rigid language. It has rules and clear principles which cant (and should not) be abused. Also, class helpers is the proper way to extend the functionality of classes through abstraction.

Anyways, here is a way you could do it:

  TEventReference = Class(TObject)
  private
    FInstance:  TW3TagObj;
    FEventName: String;
  public
    procedure   &To(EntryPoint:TProcedureRef);
    constructor Create(Obj:TW3TagObj;EventName:String);
  end;

  TControlReference = class(TObject)
  private
    FParent:    TW3Component;
    FInstance:  TW3TagObj;
  public
    function    Attach(EventName:String):TEventReference;
    constructor Create(AParent:TW3Component;AInstance:TW3TagObj);
  end;

  TSelector = class abstract
  public
    class function &Object(Container:TW3Component;ControlName:String):TControlReference;
  end;

//#############################################################################
// TEventReference
//#############################################################################

constructor TEventReference.Create(Obj:TW3TagObj;EventName:String);
begin
  inherited Create;
  FInstance := Obj;
  FEventName := EventName;
end;

procedure TEventReference.&To(EntryPoint:TProcedureRef);
begin
  w3_AddEvent(FInstance.Handle,FEventName,Entrypoint);
end;

//#############################################################################
// TControlReference
//#############################################################################

constructor TControlReference.Create(AParent:TW3Component;AInstance:TW3TagObj);
begin
  inherited Create;
  FParent := AParent;
  FInstance := AInstance;
end;

function TControlReference.Attach(EventName:String):TEventReference;
begin
  result := TEventReference.Create(Finstance,EventName);
end;

//#############################################################################
// TSelector
//#############################################################################

class function TSelector.&Object(Container:TW3Component;ControlName:String):TControlReference;
var
  x: Integer;
  mObj: TObject;
begin
  result := NIL;
  if Container<>NIL then
  begin
    controlname:=controlname.trim();
    if controlname.length>0 then
    begin
      mObj := Container.ChildByName(ControlName);
      if mObj<>NIL then
      result := TControlReference.Create(Container,TW3Component(mObj));
    end;
  end;
end;

What is this good for?

The above code does very little. It is just an example of how you can build chains of functionality using standard object-orientation. In this case all we do is to to:

  • Locate a component by name from a container
  • Bind to an event by name
  • Attach event handler to event

But it may be helpful for others to see how to do this, so I figured it was worth 5 minutes.

The syntax goes like this:

TSelector.Object(Form1,'w3button1').Attach('onclick').To( procedure ()
  begin
    // your event handler here
  end);

Note: Since JavaScript has automatic garbage collection you don’t need to worry about creating instances on the fly, they are collected and dropped by the VM as soon as they go out of scope.

To simplify things even more, we can declare TSelector as a global (unit) variable:

var
  JQuery: TSelector;

And now we can do the same thing, but using the variable name instead:

JQuery.object(Form1,'w3button1').Attach('onclick').To( procedure ()
  begin
    // your event handler here
  end);

More advanced stuff

if you want to make use of property and array references, which would be more natural to mimic jQuery and other classic JS libraries – you will have to create an instance and serve that through a function.

var
__JQUERY: TSelector;

function JQuery:TSelector;
begin
  result := __JQUERY;
end;

initialization
begin
  __JQUERY := TSelector.Create;
end;

With this in place we can add a default property, as such:

  TSelector = class(TObject)
  private
    function getForm(name:String):TW3CustomForm;
  public
    property Forms[name:String]:TW3CustomForm read getForm;default;
    class function &Object(Container:TW3Component;ControlName:String):TControlReference;
  end;

And now we can can support a more direct approach (must also alter the sub classes accordingly):

JQuery['form1'].with('w3button1').attach('onclick').to( procedure ()
  begin
    // your event handler here
  end);

Hire a Delphi superhero!

May 18, 2015 2 comments

This is your chance to hire a Delphi super-hero, namely me! That’s right, I’m presently in the market for a new full-time job!

Note: if you need people for a short project then I personally must decline, but I can forward your position at Delphi Developer (4.000 Delphi developers) and here on my website (average of 15.000 visitors per month)

I must underline the obvious, namely that I live in Norway with no intention of moving. However, I have a fully established office which has been used for three consecutive years. So I am used to working remotely, communicating through Skype, Messenger, Google Meetup’s and similar utilities.¬†This has worked brilliantly so far. Full time employment over the internet is finally a reality.

For Norwegian companies I can offer 50% commuting (100% in Vestfold) from Oslo to Akershus in the north, to Sandefjord and Larvik in the south. The reason I can only offer 50% is because I have custody for two children every other week, which means I am unable to drive long distances in combination with school and similar factors.

You need an awesome Delphi developer? Then I'm you man!

Jon Lennart Aasenden, 41 years old senior software engineer using Delphi

My experience is too long to list here, but if you are in the market for a dedicated software engineer then you have come to the right place. Simply drop me a line on e-mail (address at the end of this post) and I will get back to you with a full resume ASAP.

In short: I have worked with Delphi for 15 years, I am the author of¬†Smart Mobile Studio and wrote¬†the VJL (run-time library, similar to VCL) from scratch. I have worked for some of the biggest companies in Norway. I have good papers, excellent reputation and is recognized for my skill, positive¬†attitude and “yes we can” mentality.

Before my 15 years of Delphi I spent roughly the same amount of time programming¬†games, multimedia and contributing to the demo scene. In my view, demo programming is one of the most rewarding intellectual exercises you can engage in. It teaches you how to think and prepares you for “real life solutions” as a professional.

In my career as a Delphi developer I have written several commercial products; ranging from a complete invoice and credit management application, games and multimedia products, serial management components for Delphi, backup clients and much, much more.

No matter what your company works with, as long as it’s Delphi I’m the right man for the job.

My kind of job

It goes without saying that I am looking for a purely Delphi centric position. Although I have worked with C++ and C# and have no problems navigating and adding to such a codebase — there can be no doubt that Delphi is the environment where I¬†excel at my work.

I bring with me a solid insight into HTML5/JavaScript and am willing to teach and educate your existing staff in Smart Mobile Studio, which I am also the founder and inventor of; bringing your team up to speed with the next step in computing: namely distributed cloud computing. At which JavaScript and Object Pascal play central roles.

Torro Invoice

Torro Invoice

So if you are moving towards cloud computing or want to leverage the cloud for service oriented architectures using RPC or REST servers, I am the guy you want to talk to. Especially if you want to use object pascal and leverage nodeJS, JavaScript and the whole spectrum of mobile devices. In short, with me on your team you will have a huge advantage.

What you get is 15 years of Delphi experience; and before that, 15 years of software authoring, demo and game development on 16 bit machines.

Good qualities

I have included a small list of “ad-hoc” qualifications.¬†The list below does not serve as anything other than indicators. I have been a professional programmer¬†for many, many years and to list every single piece of technology¬†is¬†pointless.¬†But hopefully the¬†list is enough to give you an idea of my qualifications, depth and diversity.

  1. Write custom Delphi controls and UI widgets
  2. Create control packages and resource management for our projects
  3. Optimize procedures by hand by refactoring variables, parameters and datatypes.
  4. Write a database engine from scratch and also use existing engines such as Elevatedb, MSSQL, Oracle, Firebird, SQLite and MySQL
  5. Work with Windows API directly
  6. Work with freepascal and port code to Linux and Macintosh
  7. Write networking libraries, custom protocols and high-speed multithreaded servers
  8. Use classical frameworks like Indy, Internet Direct and other out of box solutions
  9. Write advanced parsers for source-code and any complex text-format
  10. Write a real-life compiler and IDE (www.smartmobilestudio.com and quartexpascal.wordpress.com)
  11. Create a graphics library from scratch, supporting different pixel formats
  12. Work with large files, performing complex operations on gigabytes and terabytes
  13. Write mobile and desktop applications using Smart Mobile Studio and FireMonkey
  14. Create your server environment, architect your RPC (remote procedure call) service infrastructure
  15. Implement custom services through nodeJS, Smart Mobile Studio, Delphi and FreePascal
  16. Use Remobjects product line: such as RemObjects SDK, Hydra and Data-abstract
  17. Link with C and C++ object files
  18. Use project management tools, scrum, SVN, github and package management
  19. Write, export and import DLL files; both written in and imported to Delphi
  20. Write solid architectures using moderns technologies: xsl schemas, stylesheet validation, rest, node services

.. and much, much more!

Note: While you may not have a need for all of these qualifications (or any one of them), the understanding which the listed topics demand that I master is of great value to you as an employer. The more knowledge and hands-on experience a programmer can offer, the better qualified he is to deal with real-life problems.

Tools of the trade

All programmers have a toolbox with frameworks, code collections or third party components they use. These are the products I always try to use in projects. I have a huge library of components I have bought over the years, and I have no problem getting to know whatever toolkit you prefer. Below is a short list of my favorite tools, things I always carry with me to work.

  • RemObjects SDK to isolate business logic in RPC servers, this allows us to use the same back-end for both desktop, mobile and html5 javascript.
  • RemObject Data Abstract
  • Remobjects Hydra
  • ElevateDB and DBIsam (DBIsam for Delphi 7-2010, ElevateDB for the rest)
  • Developer Express component suite
  • TMS Aurelius database framework
  • MyDac – MySQL data access components
  • MSDac – Microsoft SQL server data access components
  • Raize component suite
  • FreePascal and Lazarus for OS X and Linux services
  • Mono/C# when customers demands¬†it (or Visual Studio)
  • Firemonkey when that is in demand
  • Smart Mobile Studio for mobile applications, cloud services [nodejs] and HTML5/Web

Tools and utilities

  • Beyond compare (compare versions of the same file)
  • Total commander (excellent search inside files in a directory, recursive)
  • DBGen – A program written by me for turning Interbase/firebird databases into classes. Essentially my twist on Aurelius from TMS, except here you generate fixed classes in XML Data Mapping style.

Note: For large projects which needs customization I tend to write tools and converters myself. The DBgen tool was created when I worked at a company which used Firebird. They had two people who hand wrote wrapper classes for nearly 100 tables (!) They had used almost two months and covered about half. I wrote the generator in two days and it generates perfect wrappers for all tables in less than 10 minutes.

Safety first

As a father of three I am only interested in stable, long-term employments. Quick projects (unless we are talking six figures) are of no interest, nor do I want to work for startup groups or kickstarters. I am quite frankly not in a position to take risks; children comes first.

many of my products are available for purchase online

Many of my products are available for purchase online

I am used to working from my own office. I was initially skeptical to that, but it has turned out to work brilliantly. I am always available (within working hours Norwegian time) on Skype, Google, Messenger and phone.

Languages work format

For the past year I have been working full time with C# under Microsoft Visual Studio. Before that I used Mono for about four months, which I must admit I prefer over Visual Studio, and wrote a few iPhone and iPad apps. So I am not a complete stranger to C# or other popular languages. But as I have underlined in my posts on countless occations Рobject pascal is a better language. It is truly sad that Borland was brought down all those years ago, because with them they dragged what must be one of the most profound and influential languages of our time.

That said, I have no scruples about mixing and matching languages. Be it Remobjects Oxygene, C#, C++ or Object Pascal. I also do some mocking in BlitzBasic which is without question the fastest (execution wise) of the bunch, but not really suitable for enterprise level applications.

The point here being that I am versatile, prepared to learn new things and not afraid to ask. It would be a miracle if a man could walk through life without learning something new, so it’s important to show¬†respect and be remain humble. Different languages represents different approaches to problem solving, and that is an important aspect of what it means to be a programmer.

The only thing I ask is that I am properly informed regarding¬†the language you expect me to deliver in –¬†and that I receive the required training or courseware in due time. Do not expect super-hero status in C#, I am absolutely above average, but¬†it cannot be compared to my efficiency and excellence using Delphi.

Contact

If you are interested in hiring me for your company, then don’t hesitate to contact me for a resume. You may e-mail me at lennart.aasenden AT gmail dot com.

Angular JS + BootStrap and why it’s all bull

May 6, 2015 10 comments
Hype is the number of the beast when it comes to development

Hype is the number of the beast when it comes to development

Web designers and web “architects” as they like to call themselves, are into bling. Their job is after all to help bussinesses sell whatever product they have; be it a perfectly valid product or a completely useless piece of droppings. But with a good designer at your disposal you can to some degree ensure your product will sell. At least at first. So when designers and bling-doctors set out to create their tools of the trade, it’s pretty much natural that they will make it look like the greatest thing since the discovery of the wheel.¬†And people swallow it, hook line and sinker.

JQuery

Take JQuery for instance. It’s a simple little library to collection (query) information about the DOM (document object model). It allows you to set properties on collections rather than single items and it also have a handful of helper functions for effects, css and dealing with JSON and REST.

Eh.. no thanks!

Eh.. no thanks!

Yet when you read the documentation or hear people talk about it – it sounds so fantastic. You can cure cancer with JQuery, did you know that? It’s hype. It’s just spin and you should know that by now.

Angular.js

Angular is, seen from the JS developer’s viewpoint, a fantastic library. It allows you to define content in one place, controller code in another place and Angular gives you the tools to glue those two parts together. The controller is the part which has code, meaning that you access the controller to perform operations on the model (the data) – which is reflected in the view (the html). But wait just one minute! Havent we heard all that before? You have data separated from functions, functions separated from the display — yet they all come together to form a single entity? Yes. You have heard it all before. What I have just described is how a normal class works. The reason it’s divided into controller, model and view has nothing to do with better, next generation or any of that crap — it has to do with the fact that JavaScript doesnt have classes, or inheritance, or polymorphism or any of the fundamental features real programmers expect to find. So if you are a JS programmer Angular is fantastic, because it allows you to break down a presentation (read: component) into logical parts, deal with callbacks and display sequences in an orderly fashion and so on. But for a programmer coming from C++, Delphi or C# — Angular is a childish toy. There is no reason to break these elements apart, that’s why God invented VMT’s (virtual method tables).

Yeah yea, bla bla bla bla, hype

Yeah yea, bla bla bla bla, hype

I mean, have you heard some of these presentations on youtube about angular? Wow. It even get’s me going for a while there. The mental image they paint is that angular is going to solve everything! If you can just use angular, then your website will scale, dance, sing and even click it’s own likes. But when you start coding with angular you realize that it’s just a non-productive way of emulating OOP through objective thinking. It’s technical masturbation. You isolate your data in one place, your procedures in another place and your visual appearance a third place. This is exactly what OOP was created to solve. It may be a step forward for JavaScript, but compared to Delphi, Smart Mobile Studio, C++ or any “real” programming language with OOP — it’s a gigantic step backwards.

Components is MVC

MVC (model view controller) is a programming pattern, or style, which has become very popular lately. It was initially hyped by Apple to get people onboard their Objective C wagon for iOS and OS X development. Picked up by other languages, especially JS which is the mother of all bling, it has become almost the norm. Both in the world of nodeJS and DOM programming in general.

Oh but you can change things easier and use the same controller with different views, or have different controllers for the same view. So? Thats what inheritance is all about:

type

//baseclass for database read values
TBaseDBStorage = Class(TW3Object)
end;

//filtered database values, inherits from base
TFilteredDBStorage = Class(TBaseDBStorage)
end;

//basic display widget
TBaseDisplay = Class(TW3CustomControl)
private
  FData: TBaseDBStorage;
protected
  function getDBStorage:TBaseDBStorage;virtual;abstract;
End;

// display optimized for desktop
TDesktopDisplay = Class(TBaseDisplay)
protected
  function getDBStorage:TBaseDBStorage;override;
end;

// display optimized for mobile devices
TMobileDisplay = Class(TBaseDisplay)
protected
  function getDBStorage:TBaseDBStorage;override;
end;

I mean, the above OOP code allows you to fetch data from a server, and also specialize it as needed. We also roll out two display controls, one for mobile and one for desktop, and we give them different display and layout rules. This is faster and easier than writing two controllers, two data structure profiles, two html segments and two CSS rulesets. Same result, except the code above can be expanded indefinitely without messing up existing models.

So there!

So there!

What people fail to notice is that MVC is just plain old TCustomControl. An ordinary TCustomControl has data (fields), which is a model — methods and implementations which is the controller, and a view (surface, canvas or operative-system viewport). And just like under MVC the code is there to act on the data and control the visual output. What angular does is essentially to rip OOP apart under the assumption that¬†this will make things better. But it doesnt make things better, it makes things worse (!)

Mutation events

Now if you’re a JS programmer feeling you have to defend angular, please relax. I know all about the data binding layer – because just like angular Smart Mobile supports mutation event handling as well. And we do it through proper OOP with full inheritance. Mutation events is just a fancy phrase for “tell me whenever anything changes in the DOM”. So a mutation listner can be created and set to listen to changes on either the whole DOM or a particular piece of the DOM (like your panel or a button for that matter). Whenever a property change, an attribute or a CSS style — you get a callback event with the change data. Using this system Google has made a neat system for binding data changes to visual tags. For instance, whenever you change the value of a variable – you can hook that variable up to a section and have it display the change “live”. You can also bind visual input, like from a textbox, to a variable, another field or whatnot. Sounds familiar? ¬†Delphi Visual Live bindings anyone? Angular¬†creates the illusion of being black magic; and to the JS punters and kids it probably is. But sadly the only spell it casts is one of blinding hype.

Bootstrap JS

Bootstrap has to be, with the exception of jQuery, the biggest disappointment of them all. And I dont write this just to be negative or anything like that. I am genuinely disappointed at¬†how such a small library, containing nothing particular interesting at all, can get so popular so fast. Bootstrap.js is essentially a theme engine. That’s it. Yes you read right, a plain old vanilla theme engine, like we have had in Delphi and C++ for decades now. Bootstrap allows you to define simple constructs, like a panel. It will then automatically align and position your input controls and DIV’s according to a 12 point grid. Does this sound familiar? It should because every Delphi control has an align property, and TLayoutPanel has been shipping with Delphi since 2006. If you add some clever CSS styling on top of that — you essentially have bootstrap.js

Final verdict

In this post I have told you the truth about some very, very big and popular javascript libraries. They are popular and famous because for a JS developer, which doesnt have the luxury of OOP like we do – these libraries is heaven sent. But for everyone who has OOP, like Smart Mobile Studio, there is absolutely no benefit to using angular, bootstrap or jQuery at all. In fact the RTL was designed in such a way that there should not be any unknown tags in the DOM. Hence jQuery is useless because what exactly should you use it for? TW3CustomControl encapsulates the uber-hyped MVC of Angular, our web units outperforms and outweigh the angular toolbox by a factor of 10 — so again there is absolutely nothing angular provides which has any value to a seasoned and professional Smart Pascal programmer. He would make angular.js himself within days (!). Not because he is so clever, but because OOP and inheritance makes it possible. Modern OOP was designed to better help you deal with problem solving, spanning recursively through a well written and designed architecture. This is why OOP rules the world and procedural and objective programming is a thing of the past.

A better and far more powerful way

Over the next six months I will be presenting my¬†web technology. This means that the tools you will get – through smart mobile studio – and the classes you will also get, will help you create web applications that knock the boots of all these 3 libraries. So far Smart Mobile Studio has only been for mobile application development. Well that is about to change (slowly) as we begin to add more and more support for full-on “page” designs. You can naturally create websites today, but webpages demonstrates a slighly different behavior – both in terms of scaling, scrolling and general navigation. You want to create awesome websites with little code? Sure you do! You want to use your Delphi skills to create fantastic HTML5 user-controls which no other language delivers? Sure you do! And you want to connect to databases, REST API’s and your own nodeJS based service layer? Hell yes you do! Well, let’s teach these JS kids how real programmers solve these things ūüėČ