Archive

Archive for December 26, 2015

Smart Mobile, we want your ideas!

December 26, 2015 11 comments

Smart Mobile Studio has so much potential and can go so many ways, so much in fact, that we want to hear your ideas on some important subjects. As a company we want to deliver what our customers need, rather than what we feel is appropriate.

So I will be airing various topics in the weeks to come, hoping that as many of you take the time to think, ponder and give us some ideas. If you have your own framework and feel it would be perfect for global adoption, then send it to our main website (www.smartmobilestudio.com) and we will have a look at it.

Database

This has been a hot potato for a while now. It’s been a tough nut to crack because JavaScript really is utterly data-agnostic. There is no default framework to adhere to in the browser, and as such – no real object model to implement.

To make things even worse, browsers ship with 3 different DB engines (WebSQL was by far the best, but it’s now marked as obsolete), all of them so different that orchestrating one common framework to encapsulate them all has turned out futile.

But all is not lost, we still have the coolest OOP JS compiler on the market and with a solid Delphi background, we can easily create a DB framework which makes data a joy to work with. It may even be that writing our own engine from scratch is the best way to go, all things considered.

The criteria we should keep in mind is (on top of my head):

  • Support for browser engines
  • Support remote DB providers
  • JSON intermediate format
  • Easy to implement custom data providers
  • Future proof and adaptable

Browser based engines

The first point on this list, support for browser engines, is questionable. Smart Mobile Studio ships with support for WebSQL and IndexDB, and while these are supported natively by most browsers – they serve little or no purpose beyond storing name/value pairs. And they are both crippled by a ridicules 5-10 megabyte limitation.

This really is the sad truth, that browser environments at this point in time is pretty much useless when it comes to data storage. The reality is that native engines provided by the browser, gives you absolutely no benefit over a hand-sculpted, structured, indexed array. You just dont gain anything useful by using these DB engines.

SQLite

Smart Mobile Studio ships with a third engine, one which IMHO is a much better alternative than any of the native, browser based options. We took the time to compile the now industry standard SQLite from it’s orginal C source code, into JavaScript. Yes you read right, we have a pure JavaScript version of SQlite  which does everything it’s native counterpart is capable of, including creating binary compatible DB files.

But before you say “Great! Let’s use that”, keep in mind the other points on our list, namely “future proof” and “support remote DB providers”. If we marry SQLite by tooth and claw, we may end up with serious problems in the future. The database capabilities of the browser will change, just like object pascal did years ago. Remember when Paradox was the default database? Remember how hard it was to get rid of the BDE only 10 years ago? Well, if we can avoid that kindof scenario we should. By any means.

In my view SQLite should be an engine, one you can chose to use for local data storage if you need to work with complex data structures (read: data that needs SQL to work properly). It should just be one solution in a series of options, ranging from TClientDataset to TRemoteDataset (or something in that ballpark).

So already we begin to sense that our framework must abstract away any notion of where data comes from. Also, since JavaScript is purely ASYNC (including network capabilities), the link between provider and consumer must have a signaling mechanism. Requesting data and consuming data does not occur in sequence, like we have under Delphi. So we can write that down and keep it in mind as we dig deeper.

Some benefits

The benefit of picking just one engine, like SQLite, is that you can write code that pretty much knows what is going to happen. If the only engine you will ever support is SQLite then naturally your code will reflect that and you dont have to take height for scenarios which SQLite doesnt cause. So there are benefits to sticking to a simple, ad-hoc architecture. The downside is that – once data binding get’s into the codebase, throwing it out again or altering it in the future is going to be a nitemare. It may even break the codebase completely, especially for our customers.

Complete abstraction

In a perfect world data-consumers and data-providers doesnt know “how” things are done, or even what format data comes in. They only know about the access interface between them and that’s it.

From what we have covered so far, it becomes clear that in order to deal with multiple sources of data (local database, remote access) that at least on this level, abstraction is a must.

In my mind, at least 3 source types should be supported out of the box:

  • Remote DB
  • Local DB (WebSQL, SQLite drivers, TClientDataset)
  • UDD (User defined data)

What we end up with is an architcture very much like the .net framework. A framework which involves logical, linear naming as such:

  • DataEngine (*1)
    • DataConnection (*1)
  • Dataprovider
  • Dataset
  • Datasource
  • Databinding
    • DataEmitter (*2)
    • DataConsumer (*2)

*1) A data connection object contains properties and methods for executing and maintaining a connection to a remote host. Typically this object is exposed as a property of a DataProvider, which in turn represents a particular engine.

*2) Internal mechanisms used by visual components as couplings to a data-binding. The binding itself does little but describe source and target, while emitter and consumer objects are created on each side of the binding when the binding is activated

Data provider

What is a data provider? It is an object which provides access to structured data. It can be a table, the result of a query, a fixed user defined list of values or anything that can be represented as a linear, structured set of data.

Dataset

Like the name implies this is a set of data, a collection of items or rows -where each row has a series of columns, and each column can contain data of a particular type.

Datasets are exposed by dataproviders, they can be bound or unbound. Bound datasets are live and any update/write operation is performed on the database itself (with corresponding update to any DB aware components). Unbound datasets have no such contract and must be manually updated.

Data source

This is the bridge between the non-visual and visual components. A data-source keeps track of a collection of data-bindings and replicates update signals to attached components.

Databinding

A databinding is an object which describes a connection between a visual control (textbox, boolean switch etc.) and a dataset field. Whenever the visual control alter the value, the change also occurs in the datasource. And whenever the value is altered in the datasource, the databinding makes sure the value is also updated in any bound visual controls.

When a databinding is activated (for instance when a dataset is populated by data, like a table being opened), it creates two auxillary objects: A data emitter and a data consumer object. The emitter and consumer is created as child objects for the binding itself, but the visual control is given an interface to the consumer so it can read and write data through that.

Visual programming, extending the IDE

All of the above is great and it will no doubt give us the same benefits as .net developers currently enjoy. But in order for databases to become a living part of the Smart Mobile Studio cycle, we need to extend the IDE.

The first option is to do like Delphi does it, and allow non-visual objects to be dropped onto a form. But this is truly a complete waste of CPU power and to be honest. It can be done (actually quite easy), but I have nothing positive to say about it. It opens up for a world of pain. People are lazy and will copy stuff between forms rather than isolate data access in one, central place. Trust me on this, it would be to take a step backwards.

A much better idea is to create a new entity. A visual component, but one that can only be dropped onto a special area of the form designer. This gives us plenty of benefits:

  • We avoid the overhead of TComponent
  • All objects can be created by the automatically generated code
  • Lightweight and fast, avoid code bloat
  • Easier for us to implement in the IDE
  • Easier to add to the existing package format

Here is a picture (example only, ignore colors) of what I have in mind. As you can see we have a bottom horizontal region where non-visual components can be dropped. When you select them you get to edit their properties in the property-inspector, just like under Visual Studio.

Globals and locals

In the above idea we find an interesting concept. First is the obvious, namely that the generated startup-code for a form creates the non-visual components when the form is created. But we can also introduce a global scope drag & drop placeholder (!)

dragAndDrop

A non visual drag & drop area like Visual Studio has would be nice

The global placeholder, which becomes available when you select the application unit, would be automatically created when the application starts. This is actually very powerful. It means that we can isolate our central DB components here, and they are compiled into TApplication. They would be globally available to the entire application.

The benefit is that on form level, all we have to drop is a datasource and then X number of bindings linking to visual components. See?

And it doesnt stop there, once we have stuff like image-lists implemented, they too can be make globally available by keeping them in “globals”. I know it’s not the same as a TDatamodule, but when it comes to DB programming it’s actually a restriction that helps us write cleaner code. You wont believe some of the horrible DB code i have seen over the years. People copying queries into every form, bloating the application and abusing the database, rather than isolating all data access in a single datamodule.

Your ideas matter

I would very much like to hear your ideas around this. And please, spend a few minutes reflecting on the model. And if you have a model presently in use that you feel SMS would benefit from – then dont hessitate to present it. The goal here is to create a solution which is future proof, easy to work with and adaptable. Not “who is the coolest programmer”.

I also think it’s important to have a closely knit, open and friendly community — hence i share and write as much as I do. I’ve never been a fan of closed doors or companies that keep their friends at arms-length.

Well, have a happy XMas and let’s make 2016 a kick ass Smart Mobile Studio year!