What about having ability generate app to solve & automate any tasks?

Integration of IT products, automation in businesses and goverments moves forward but seems it might go much faster. I think we can boost the process by making software products available or even free; ready for use; easily maitained and extended. So whats about having the dynamic system that are ready to use as a different of types of software like ERP, CRM, Project Management and many others.

Notice

  • In the post I will use table / record (SQL) and collection / document (NoSQL) interchangeable.

High level system structure

The dynamic system may consists of different apps with a different purpose. Like the company may have CRM app, equipment management app, project management app, warehouse management app and many others.

Components High Level

  • The dynamic system may look like a simple web app. The most relevant concept of UI is Github/Gitlab with Repositories; Confluence with Spaces; Jira with Projects.
  • App is a main component of the system that’s intended to solve business problem.
  • Configuration is a description of the app structure & business logic.
  • Data is a just a database.

Dive into Configuration and Data

The first component of configuration is Schema. As app created, the user is able to configure app - edit app schema.

Component Schema

Schema defines all tables or collection that the app have. Looks like we can implement different UI for creating and editing schema: regular input forms, UML editor. Field is a regular enitity of the databases. Field has name, type, default value. We can also define fields like createdAt, updatedAt with autogenerated value.

Let’s summarize what we have with use case diagram: Use Case Basic

We’ve got a dumb alternative for MongoDB Compass or pgAdmin. Looks not so inspiring, but let’s go deeper :-)

Actions

Let’s add a new one components into configuration - Action. The action is a lambda function that might be applied to a record or set of records in specified data collection. Action components:

  • name
  • collection reference
  • a piece of code that implements some business logic

Component Action

Example:

Let’s image we’re making a simple shop app with a following data schema. Manager of the shop would like to have a button “Execute Order” for an object of the order. So, here’s an example of the action: we’re creation an action object that points to Order collection (the action might be applied only for a record of collection Order). A piece of logic that we’d like to implement: mark status of the order as “done” and withdraw ordered count from warehouse.

Example-Shop

Based on example lets think about how it can be implemented. The most relevant implementation comes from AWS Lambda’s.

def action_handler(object_id):
    // here might be rest api request to the system to get object by id
    // here might be rest api request to update object
    // here might be rest api request to delete object

The idea of writing a code is not so user friendly, but it can work. Another idea - visual programming - something similar to Unreal Engine Blueprints, Google Blockly, Python Open Source Visual Programming. Seems we can easlily wrap all db function into blocks as they have a strictly defined interface and additionaly enable rest api request from this flow.

Data Flow Programming

Looks much simpler.

Again, let’s extend use case diagram:

Use Case Action

Dashboards

For now exists quite important problem: user can have access to all collections, but when collections have links user can see only ids instead of user friende link decscription. Example:

Dashboard is one or several(aggregated or joined) collections that might be displayed in different forms: table, plot, chart, 3d and many many other ways.

Component Dashboard

Example - warehouse table:

id product_id count
12 1 1
14 3 1

Example - warehouse dashboard - aggregated warehouse and product collection in a table view:

id product count
12 orange 1
14 banana 1

Again how to define aggragation or join rules for dashboard - either lambda or visual programming.

Visualization: support it’s reasonable to add visuazation setup param as param of the dashboard. Visuazation setup may look like:

  • visualization type: grid, chart, plot, calendar, gantt chart.
  • visualization param configuration - we need to select what parameters use as input to visuazation.

Inspiration for the charts were took from MongoDB Charts

Updated use cases:

Use Case Dashboard

Rest API

I really hope that you’re still able to imagine the system I described. The rest of the system will be very briefly describe.

The backend of the system may be implemented as REST API. I’d like to divide API into 2 parts:

  • Configuration API - static API that Application has
  • Data API - dynamic or template API - generated based on configuration

Configuration API example:

POST /v1/configuration/schema/make_structure
GET /v1/configuration/schema/structures
GET /v1/configuration/schema/structures/:id
PUT /v1/configuration/schema/make_structure
DELETE /v1/configuration/schema/make_structure

Data API example:

GET /v1/data/:collection_name/
POST /v1/data/:collection_name/make_object
GET /v1/data/:collection_name/:id
PUT /v1/data/:collection_name/:id
DELETE /v1/data/:collection_name/:id

As we’re having API it’s possible to use the system as backend for IoT devices.

Forms

What if we’d like to have one page with a big form that may reference or create object in multiple tables. The most suitable idea is to have a UI form & handler - like action - on backend. The handler for the form will divide input params into objects and put it into different tables.

Pages

Pages is just about UI. What about to unite some forms and dashboards into one web page? Looks like easy idea. :-)

User management

This point about user and access management wasn’t mentioned above but it’s really important point. I belive that we can avoid implementing complete user management solutin in the system. It would be enough to integrate any available IAM - keycloak for example.

Pipelines

A concept about having lambdas that has access to the data and may execute implemented logic. The pipelines might be triggered manually, automatically by some rules. Usage: monthly, nightly reports; data cleanup; any automated logic. It’s like action from above but without refecence to an object. Pipelines might be usefull on integration with IoT Devices to monitor states of realworld system; analyze data flow.

Problems

  • Configuration Versioning. For now I didn’t dived into the question how to manage version and safely update data from version to version in case of changes in configuration.
  • Public API. This probles comes from the previous. What if we have integration between out business entities and one of the chage configuration. Data API might be changes, so other business entity must know the issue and update their business logic.
  • Hardware Integration. How to connect usb bar code scanner to the system or camera in the most efficind way?

Missed concepts

  • Rules & data access management
  • File storage

Final idea

As we have a system to generate other system we can easily create a different predefined configurations (templates) of the system. Finally, usage flow for using our dynamic system would be select a template and use. CRM, Project Management, Warehouse Management Solutions, ….

Summary

To the end of writing all these ideas and concepts I come to a more general naming for components of the system:

  • Low code backend
  • Low code and partly generated frontend
  • Application Template Library

I haven’t dived into researching market about such systems but I believe having such system open source might really boost integration it goverment and businesses.

I hope someone was able to reach the end :-) Any ideas on it?