Home > Language research, Linux > Introduction to N++, a process oriented programming language

Introduction to N++, a process oriented programming language

December 19, 2014 Leave a comment Go to comments

For some time now I have been working on implementing various programming languages; writing parsers, sub parsers and dictionaries. To date these languages have been well-known languages, like pascal, visual basic and (to some extent) a subset of C#.

Programming languages are typically born out of two schools of thought: pure necessity, or just good old fun. I remember reading about the E programming language when I was a teenager for instance, which is a language that looks quite impressive – but serves little purpose. To be blunt: it doesnt give you anything in particular in terms of actual tools or advantages over, say, any other language out there.

And then there are languages which are just a complete waste of time, like brainfuck and it’s derived mathematical madness. If your idea of fun is jumping butt-naked into an inverted number generator (read: PI) – then by all means, brainfuck is your language. But for the rest of us — perhaps something more tasteful and useful is in order..

Presenting N++

N++ is a language I am designing at the moment (N stands for Nandi, Shiva’s ox), or adding partial support for in the Quartex IDE. While I cant cover everything in a single blog post (and certantly not at this hour), I can present some concepts you may find interesting.

N is a language designed to collect, process and distribute data in large quantities; hence the strong power of the oxen Nandi. It belongs on the server, although there is no reason why you cant use it from inside a desktop application or as a service.

Let’s start with the classical hello world:

program ("hello_world") {
  handshake {
    input  { void; }
    output { void; }
  }

  execute(*) {
    writeln("hello world");
  }
}

Let’s look at the program in broad terms. Since most of you are programmers you probably have some idea what is going on in the above code. The [pre]program()[/pre] block defines that this textfile is indeed the program file (as opposed to a module). All programs can only have a single program file, but as many modules as you like.

Next comes the handshake. Now this is a very handy (pun intended) subject. What it does is define the input the program can expect (or demand) as well as the output. In the above example we dont expect anything and we dont deliver anything, so both input and output is set to void.

The heart of this little program is within the execute block, where we use the method “writeln” to output the text “hello world”.

If you are wondering what the * character means inside the brackets, that section is called the data-view. In essence the * character means “what is known to the parent object”. In this case the parent is the program itself, which means that all global variables and objects should be regarded as known (or inherited into the scope).

Feedback loop

While hello-world is always fun, let’s do something a bit more complex. Let’s write a program that accepts X number of text arguments from the command-line, and then prints it back out again:

program ("feedback") {
  handshake {
    input {
      string arguments[];
    }
    output { void; }
  }

  criteria (*) {
    arguments <> null;
    arguments.length > 0;
  } execute {
    process (arguments,item) {
      writeln(item);
    }
  }

}

As you can see this example defined a typed handshake, where it expects to receive an string array as input. Since the program or module doesn’t produce anything, the output is again set to void (meaning “nothing”).

The criteria() section is probably the closest thing you will get to an IF statement in N++, what it does is validate X number of conditions, if they result in true the execute block is executed. You can also have an optional “fail” block to deal with scenarios where the criteria are not met.

The process() method is N++ variation of a FOR/NEXT loop. What it does is process the content of an object one-by-one and apply whatever code is inside it’s block on the data. As you can see from the parameters the arguments array from the input is the data we want to process, and as our second parameter we have a variable representing the current item.

Chainable criteria

One of the cool aspect of N++ is that you can chain these code blocks in various patterns. For instance you can chain process() and criteria() as such:

criteria {
  /* do not execute unless list has items */
  list.length>0;
} process (list,item) {
  /* do something to each item in the list */
} fail (e) {
  /* Criteria not met */
  writeln(e);
}

Or, apply criteria per item:


process (list,item) {
  writeln(item);
} criteria {
  /* Skip all items which does not begin with "jo" */
  item.firstname == "jo*";
}

Dynamic structures

While looping through data is fun, it’s not really something new. But being able to shape known data from various sources into new structures is extremely handy, especially for web services. Again it’s not a novelty – but here it’s a fundamental aspect of the language itself:

module ("data_export") {
  handshake {
    input  { void; }
    output { object[]; }
  }

  define dbConn as database("myDB@localhost:8090;user:admin;password:adminpassword");

  execute(dbconn) {
    collect ["users,"info"] from dbConn as data;
    set output = build(data,smAscending) {
      uid:      users.UID;
      username: users.userHash;
      password: users.passHash;
      fullName: info.fullName;
    }
  }
}

The above introduces a couple of new concepts. First there is the “define” keyword, which defined a data-source. In this case we associate “dbconn” as a database which we can obtain information from. By default N++ regards all types of data as either a single object, an array of object or a dictionary of object. You may also use an array as a dictionary – at which point the array object is transformed into a dictionary.

The collect() method does what the name implies, namely to collect data from a data-source, in this case we grab all the records from two tables (users and info) and stuff those into a named container (data).

And finally a magic method, namely the “build” function. In this case we create a completely new structure which will be sorted ascending — and then we map fields from the data-source(s) into the new structure.

When emitting JSON what this module returns is:

[
{
"uid": "1239094",
"userName": "AF965BF1274CCE1",
"passWord": "C009A6487BC120E",
"fullName": "Jon-Lennart Aasenden"
},
{
"uid": "987351",
"userName": "BF982737CD00A92",
"passWord": "AF965BF1274CCE1",
"fullName": "John Calvin"
},
{
"uid": "670941",
"userName": "09F9CA1BF982737",
"passWord": "4CCE1986CDE00AF",
"fullName": "Dave Jones"
}
]

The general idea here ofcourse, is that threading and multi-processor programming should be equally simple. If you have 10 million records and try to join them into a new structure like above, chances are it will take forever before the method returns. In fact, it may even break your database. But for ad-hoc amounts of data, threading it makes sense:


  background-execute ("user-export",tpIdle, => (*) {
      /* Anonymous procedure callback handler */
      signal.send([*].process.owner,MSG_NOTIFICATION,"$DB-EXPORT-READY");
      } ) {
    collect ["users,"info"] from dbConn as data;
    set output = build(data,smAscending) {
      uid:      users.UID;
      username: users.userHash;
      password: users.passHash;
      fullName: info.fullName;
    }
  }
}

In short: when the data export is done and the data is delivered as the module output, a signal is sent to the owner process, of type “notification” with a string message saying that the export is complete.

Expanding the idea

The idea behind N++ is that a language designed exclusively to process data, working with stores (which in this case can be any linear flow of data-items, such as a database table, a folder with files, a text-list or any other vertical media) will be able to take shortcuts traditional languages cannot.

As of writing, N++ is barely out of the idea stage, with only the bare-bones parser and AST implemented, but already we are able to cut corners that are, in computing terms, quite costly.

Take the “IF” statement for instance. In any procedure involving more than 2 variables, you end up writing criteria testing at the beginning of the procedure:

function TMyService.DoSomething(a,b,c:Float):Float;
Begin
  if (a>1.0)
  and ( (b<1.0) and (b>0.0) )
  and (c>0.0) then
  begin
    result:=A * B - C;
  end else
  Begin
   //raise exception
  end;
end;

Which can be more cleanly expressed through n++ direct approach:

method doSomething(demand a,b,c) {
  criteria {
    a > 1.0;
    b < 1.0;
    b > 0.0;
    c > 0.0;
  } execute {
    output float(a * b - c);
  } fail (e) {
    /* express error */
  }
} breach {
  /* parameter contract not met */
}

As you can see from the above code, N++ have some new concepts such as contract based interfaces. The interface of a procedure defined the syntax and expected input (as it does in all languages). This must not be confused with COM interfaces, as interface in this context is refering to header (as in C’s .h file). Object pascal doesn’t need a separate header file like C, since the syntax provides an interface section at the top of a source-file.

N++ doesnt even need that, and it can even (much like javascript) allow calling methods and modules without correct parameters. This is called a breach-of-contract and can be caught much like any error.

Contract based programming is not just limited to data-mapping, but is also used in parallell programming and cluster programming where invocation of a method can be done across the domain. All invocations are actually sent as signals within the cluster and workload can be distributed across several computers (nodes).

While that is not something I have even begun sculpting, putting it into the foundation of the language is important.

Well, I’m off to bed — it’s been a long day.

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: