Convention For Command Line Interface Commands

Many of the modern cloud services provide a Command Line Interface (CLI) tool for users to interact with the services instead of an API or a GUI. I have written another post with a title UI and CLI Are Equivalent!: it basically states that UI interaction should be possible to directly convert into CLI client commands. This would allow more explainable interaction and also would enable converting GUI actions into CLI commands and ultimately into API interaction. Unfortunately, CLIs lack strong conventions which would drive evolution towards correct direction i.e. making the UI interaction convertable into CLI commands.

To improve interoperability, I am proposing convention for CLI commands. In the simplest case it is about creating more structure and stricter format:

cmd mandatory-args mandatory-flags mf-args

In short, any CLI command must be followed by mandatory arguments and mandatory flags (if any). There can be also sub-commands, optional arguments, and optional flags with arguments.

cmd [sub-cmd mandatory-args optional-args mandatory-flags mf-args optional-flags of-args] mandatory-args optional-args mandatory-flags mf-args optional-flags fl-args

Args can be 1) mandatory command args 2) optional command args 3) mandatory flag args or 4) optional flag args. Args are positional, but there can be named args which must be after the positional args. Positional args are space separated strings and named args are key=value pairs. An argument can also be an array and then the array items are comma separated e.g. “a,b,c,d” or “k1=v1,k2=v2,k3=v3”.

Flags themselves should have the common formats: long (–long) or short that is short version (-l). Also, case sensitivity holds: lowercase and uppercase flags are different! Long version should match to a real word and short version should match some combination of characters of the long version – preferably first character/letter.

P.S. AI like LLM loves structure, and you need to prepare your API and CLI to be ready for AI-to-AI (A2A) communication (cf. Machine-to-Machine communication aka. M2M). Having well structured format for CLI helps with that.

Schema For JSON ala LDAP

Those of the a bit older software professionals know LDAP. Due to a work assignment I had to refresh my mind on it. While working I had the idea that “What if we use LDAP schemas also for JSONs?” I mean there is definitely some common ground, and JSON could benefit from all the work that has been poured to LDAP over time. Also, previously I have written about a new and simplified future JSON format called JKON where arrays are basically deprecated. My idea is quite simple: for a LDAP entry there will a JKON entry. LDAP-style schema means that for an JKON object there are some attributes/fields which are mandatory (LDAP MUST) and optional (LDAP MAY). Also, the simplest schema rule is that an object must have at least one class and unique name defined! IMHO, LDAP-style schema actually fits quite well into JSON-style objects because for each object you defined to what classes it belongs to and values are written as fields / maps straight into the object itself! There are a lot of well-defined LDAP objectClasses and attributeTypes available, and it is quite easy the define your own custom extensions aswell.

# LDAP entry
dn: cn=John Doe,dc=example,dc=com
cn: John Doe
givenName: John
sn: Doe
telephoneNumber: +1 888 555 6789
telephoneNumber: +1 888 555 1232
telephoneNumber: +1 888 555 1299
mail: [email protected]
manager: cn=Barbara Doe,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top

And the corresponding JKON entry

// dn => un (unique name)
// objectClass => class
// multivalued entries => fields of a nested map
{
 "class": {
   "1": "inetOrgPerson",
   "2": "organizationalPerson",
   "3": "person",
   "4": "top"
 },
 "un": "cn=John Doe,dc=example,dc=com",
 "cn": "John Doe",
 "givenName": "John",
 "sn": "Doe",
 "telephoneNumber": {
   "1": "+1 888 555 6789",
   "2": "+1 888 555 1232",
   "3": "+1 888 555 1299"
 },
 "mail": "[email protected]",
 "manager": "cn=Barbara Doe,dc=example,dc=com"
}

New Idea: Integration With Logs and Events

The good thing with APIs is that you can allow your API client to implement whatever logic it needs and thus keeping your core product clean and focused. Event integration leverages the same idea but reverses the direction: any event in the system can be sent out to APIs which have registered as listeners.

First, I will explain the idea as a whole because it might be hard to grasp at first: it is like centralized logging service but instead of receiving and processing log messages, it will receive, process, and send (i.e. dispatch) any types of structured events (i.e. messages). Any service can join the network of loosely connected service: the central service listen to requests to publish message (publish of pub-sub) and requests to subscribe messages (subscribe of pub-sub). Any joined service can send events to dispatcher that then transmits them to other services which have expressed their interest to those types of events. A very close analogy of this service would be, as said, a centralized logging. The benefit of this kind of a service would be that any service owner can integrated their service plugin-style. Basically, Zapier is one, albeit simple, example of wide set of possibilities which could be achieved with log and event integration… Please read also about difference between event-driven and data-driven integrations.

Traditional integration means integrating (software) systems by enabling exchange and transformation of data (messages) between multiple different communication end-points. Often integration logic is written in form of (integration) applications. It consists of routes and components along each route. For instance, Enterprise Application Integration (EAI) patterns and corresponding components are typically available in integration products like WSO or BizTalk.

EAI
http://www.enterpriseintegrationpatterns.com/patterns/messaging/index.html

What I am proposing is something different: integration with logs. Wait, what, logs? Inter-process communication (IPC) has a long time one of the standard ways of exchanging data between multiple processes: e.g. to and from a file. Also, modern software technologies like containers have been using files for example in monitoring: centralized (structured) logging can be used for monitoring and analytics. My idea is that processes write text to loggers which forward it to a centralized logging service that processes the (log) messages. This part is just typical modern logging…

  • Writing logs can be done in different formats but JSON seems to be a good choice because it allows loose structure and better processing in later stages
  • JSON entries can be stored in (scalable) schema-less (NoSQL) database like Couchbase
  • There are many powerful tools to make queries over massive amount of data store in NoSQL like Elasticsearch. The possibility to search and query data is very important driver for log-integration!

I still don’t understand the idea! Logs can be used to store data that does not need to exist with the application itself, like audit logs or trails. Basically log entries could also be understood as events and therefore the terms are used here interchangeably.

  • Target system must have API to listen to events. Basically this means that the system exposes an API, which is quite typical in modern world!
  • In event-integration, the target system must also be able to subscribe (pub-sub) to any new events i.e. subscribing to an integration (with the help of service discovery?)
  • There must be a service for querying all types of events, and the target system must know where this service is located (service discovery)
  • Any source system must be able to listen to new event subscriptions and register them, and publish all subscribed events to its subscribers!

The main case for this type of integration would be automation. Often digitalization means the ability to expose an API and react to received events. I think that there is much more general pattern behind this: you guessed it “event-integration”. If publishing – and subscribing to! – an event can introduced as plugins the interactions between systems would become much more easier and faster to create. A fun example: marketing automation was born because someone noticed that there was too much manual work with repetitive tasks and realized that certain triggers (i.e. events) can be digitalized! These events are points in customers journey in a service they use. Note that this is closely related to (orchestrated) distributed transaction pattern called Saga!

Wizards: Designing And Implementing Callback State Machines (CSM)

Wizard Flow
Figure 1. Wizard Flow

I have introduced earlier a model for programming with finite state machines (FSMs). The ECAS-HASM model provides an intuitive and comprehensive way of constructing programs and reducing complexity in general. Now it is time to make some refinements to the model because I realized that GUI Wizards should also follow the same idea – loosely. This idea is depicted in the Figure 1 in which the main structure is presented.

First, let’s take a step back and think about how computer programs start. Main Class is something that every program should adopt: it is the first step on the path of realizing that sequential execution of code is best suited to the lowest levels of executable code. Why? Because real world is not strictly sequential and it is really difficult to perfectly model it: it contains all sorts of contexts, events, I/O etc. Let’s recap the relevant parts of the idea here:

  • Main method in main program loads the whole program progressively (loading the special Main class is equivalent to “public static void main”)
  • Main program configures the program (it will be important to have life-cycle events to which plugins can join as listeners/consumers)
  • Main program initiates “consciousness loop” (this loop or other concurrently running parts of the program run as long as the program itself does)
  • Main program stops and exits after the initialization and configuration of the whole program including the forked processes have been done (i.e. the main program is guaranteed to stop/halt in relatively short period of time!)

After the Main class has loaded the computer program, the ECAS-HASM model kicks in. The implementation of the model is responsible for running the program. Now let’s assume the program needs the ask a user for a username and a password (I/O). Often this done in a GUI prompt, or a dialog. I already have wrote that dialogs are basically wizards. This post is mainly about defining a proper flow model for the wizards: the above picture shows a finite state model for it. So the reason for talking about Main class and ECAS-HASM here was that since they already exhibit similar behavior they make it easier to understand how CSM works: all of this is a part of the bigger picture, and after having realized it convergence emerges naturally.

In the CSM model there are states, state transitions, and related events which get triggered on certain Actions or Conditions. The image should be quite self-explaining, but I will try to explain the most relevant parts next. One of the main ideas is that every dialog, or wizard, or flow must be defined in detail: no short-cuts are allowed initially! For example, a flow typically has to implement event listener that can act based on the event (six of them) and both source and destination state! Also, as you might have noticed a flow can be stopped at either initial or final states (black circles), which means that a user in the middle of a flow might not be able to immediately cancel the flow: he would have to press “Back” button multiple times and even after that hit once “Cancel” button – you don’t have the possibility to close the wizard window from the upper right corner…

From the programmers point of view this model makes things easier but albeit more tedious. It is easier to implement wizards which can be resumed and persisted. Also, this approach fits well with asynchronous computing. What is also interesting is that Java8 CompletableFuture allows resuming code execution after a it has been paused (e.g. result is not available immediately) at some time in the execution history! Combining (or chaining) functions is a very powerful feature that we could utilize in a practical implementation with Java8 and Lambdas. Together with a library that can “Create state machines and lightweight state machine-based workflows directly in java code” we could start implementing proof-of-concept immediately. EDIT 2018: Spring State Machine project brings SM thinking to also to Spring framework / ecosystem…

How I See The Future Of The Web (Part 2/2)

In my previous post I discussed about future applications and discussed if the standalone apps were coming back. Actually what I think is that (web) applications become device independent and transparently fitted and optimized for each device. Web applications will replace web browsers and they are loaded directly from the internet dynamically with no manual work required. This means that future applications enter into era where they just simply exist in one place. In the bigger picture this means that the role of the applications themselves becomes less important: focus will be on the interconnected services and how well they fulfill the needs of the customers with excellent user experience – delivered customer value is the key.

  • Client-Server model is still valid, but individual servers will be replaced by true cloud, cf. immutable and serverless architecture.
  • Every application becomes connected, but offline feature with caching is still a strong and sometimes a mandatory requirement.
  • Business logic will reside on server side still, because service owner manages that side: however clients will become much smarter with help of APIs.
  • Traditional web browsers start to disappear because it will be obsoleted by modern “standalone applications”. For example, composable universal apps.
  • Traditional web applications (responsive HTML5+JS+CSS) will disappear along with the browsers: the number of possible combinations of mobile devices, operating systems and OS versions, browsers and browser versions is “just too much”.
  • Mobile applications will disappear too because they are device dependent which means that producing, testing, maintaining them is expensive and time consuming. This happens when better technology emerges.
  • AI will be integrated into the applications and services, and it will be heavily utilized! For example, automating end-users’ tasks and improved device IO for typing, annotation, etc.
  • End user devices are mobile, but apps can adapt to any device. Security issues become even more challenging.
  • Easier and more productive programming for UIs, which leads the less costs, faster delivery, and better UX.

In short, browsers are not the way to go, but browser-like apps are. Every device especially mobile device have “desktops” too! They can be searched and installed into a sandbox from public world-wide application service catalogs (repositories). Apps provide localisation and other services to the apps which in turn are integrated to multiple services. GUIs are made with Designer-tools and programming will be more productive, faster, and less costly. This will be also the key to build a bridge with apps and both Virtual Reality (VR) and Augmented Reality (AR) UIs – the future standard of end-user interaction.

But what about native mobile apps? A Techcrunch study shows that only 5 apps are used heavily on daily basis. Further a survey by Google reveals that only 1 of 4 apps are found via search engines. I think that the problem is that most apps do not deliver enough value to the user and on-boarding is too cumbersome. What this means is that “world” is not ready. There is both room and need for improvements, and within 50 years we will see Moore changes than you can think of – pun intended. But first steps we can take already now

New Idea: Rapid Application Development Framework For Java (RADFJ)

// RADFJ
public void doItBusinessMethod(Collection <InputMessage> im, Collection  <InputMessageContext> ic,
                               Collection<OutputMessage> om, Collection <OutputMessageContext> oc);

Rapid Application Development Framework For Java (RADFJ) would provide rapid prototyping with the help of uniform component design and component composition. This means that each component processes incoming messages and routes each message based on routing rules. An application is simply a flow where messages travel through sequences of components. Implementing custom components is easy: just implement the interface shown previously.

High-level methods with integration interfaces would implement the interface. Input and output messages could be beans/POJOs so that message properties and structures can be defined properly. Anyways, the idea is that an application consists of components with input ports and output ports: then we can compose applications by chaining multiple components by just defining targets (for certain outcomes) from some set of sources. How this is rapid? It is because developing independent business methods i.e. components that can be put together loosely couples logic which is typically easier for the programmers to digest. Also, routing rules are easily updateable and they are quite intuitive way of representing the flow.

This type of rapid prototyping could be use together with REFT to build integration capable applications that also work in system level. We could also take these ideas and combine them with state machines: what if we combine ideas of both building applications graphically with state machine like approach like in ECAS-HASM?

To summarize what RADFJ is. It is framework where business methods simulate integration components, messages (with context) simulate messages that flow between the components, routes that simulate chaining of methods, and where application is actually captured as a set of different flows.