All the code presented here is available at github.
Overview
In Critical Manufacturing we’ve invested a lot of time and effort in making sure most use cases are covered with the use of configuration and metadata. From an extensive low code platform, to master data, our product is highly configurable and dynamic.
In this blog post we are going to delve into a more complex scenario, where we build a low code task that has a UI component that can’t be described with just metadata.
Currently, Connect IoT offers a task with a typescript code sandbox. As our analytics use cases grow more complex, python becomes the defacto standard for data orchestration and transformation. Luckily with tools like Pyodide it’s relatively simple to incorporate it into Connect IoT.
The goal is then to build a Connect IoT task which offers a UI with Python code sandbox and is able to run python code and import python libraries.
Scaffolding
The CM CLI is the entrypoint for all customization scaffolding. For most iot implementations where we don’t require any UI customization we would just run:
cmf new iotIn our scenario, we want to create a customization task that is a hybrid between nodejs for the execution code and an Angular component for the rendering in the low code designer. Therefore, the scaffolding is slightly different as we are not just creating a nodejs project, but also an angular one CM CLI IoT Scaffolding.
In version 10 upwards, when creating a ask with a full on angular customization, there’s a dependency between the HTML customization package and the IoT customization package. Therefore, before you scaffold, make sure you have an HTML package already scaffolded.
In order to generate a new package we need to specify the html package location and if it’s an angular package:
cmf new iot --isAngularPackage --htmlPackageLocation ./Cmf.Custom.MESProject.HTML
We can now generate our angular library that will hold all our tasks and converters.
npx @angular/cli generate library @criticalmanufacturing/connect-iot-controller-engine-python-tasks
The library boiler plate code alread starts with an example task, but let’s create our own by running:
npx @angular/cli generate @criticalmanufacturing/ngx-iot-schematics:task
In our HTML package we can now finish up by adding our custom package and adding it in the related packages:
npm i ../Cmf.Custom.MESProject.IoT.Packages/dist/connect-iot-controller-engine-python-tasksand in the cmfpackage.json of the HTML package:
{
"packageId": "Cmf.Custom.MESProject.HTML",
"version": "1.0.0",
"description": "Cmf Custom CMFLabs Cmf.Custom.MESProject.HTML Package",
"packageType": "Html",
"isInstallable": true,
"isUniqueInstall": false,
"contentToPack": [
{
"source": "dist/cmf.custom.mesproject.html/**",
"target": "",
"ignoreFiles": [
".npmignore"
]
}
],
"relatedPackages": [
{
"path": "../Cmf.Custom.MESProject.IoT/Cmf.Custom.MESProject.IoT.Packages",
"preBuild": true,
"postBuild": false,
"prePack": false,
"postPack": false
}
]
}Finally we can import it by adding in the app.config.ts:
(...)
import { Metadata as CustomTasks } from '@criticalmanufacturing/connect-iot-controller-engine-python-tasks/metadata';
export const appConfig: ApplicationConfig = {
providers: [provideRouter(routes), provideServiceWorker('ngsw-loader-worker.js', {
enabled: !isDevMode(),
registrationStrategy: 'registerWhenStable:30000'
}),
provideMesUI(
{
tasks: [
CustomTasks
]
}
),
provideMetadataRouter()]
};Now run cmf build and you will have both the HTML package and the IoT Package compiled and ready to go.
Developing
With this scaffolding we can now start the local environment.
Using the vscode task stack-up and running npm serve in the html package. We are able to import our task in the UI and to build our customization.
Project Folder Structure
connect-iot-controller-engine-python-tasks/
├── .eslintrc.json # ESLint rules for linting TypeScript/Angular code.
├── .gitignore # Git ignore patterns for this project.
├── README.md # Project-level documentation (this file).
│ # Packed npm artifact produced by packaging.
├── ng-package.json # ng-packagr config for the designer bundle output.
├── packConfig.json # CMF package type definition (TasksPackage).
├── package.json # npm metadata, dependencies, and build/test scripts.
├── tsconfig.lib.json # TypeScript config for Angular library build.
├── tsconfig.lib.prod.json # Production variant of library TypeScript config.
├── tsconfig.lib.runtime.json # Runtime (CommonJS) TypeScript build config.
├── tsconfig.spec.json # TypeScript config used by Angular test tooling.
│
├── metadata/
│ ├── ng-package.json # ng-packagr config for metadata package output.
│ └── src/
│ ├── public-api.ts # Public exports for metadata package.
│ └── lib/
│ └── metadata.ts # Connect IoT package metadata (name, load, tasks list).
│
├── src/
│ ├── public-api-designer.ts # Designer-time entry point exports (module/designer/settings).
│ ├── public-api-runtime.ts # Runtime entry point exports (module only).
│ ├── public-api-runtime.js # Compiled JS for runtime entry point.
│ ├── public-api-runtime.d.ts # Type declarations for runtime entry point.
│ ├── public-api-runtime.js.map # Source map for runtime entry point JS.
│ └── lib/
│ └── tasks/
│ └── python-code/
│ ├── README.md # Detailed documentation for python-code task folder.
│ ├── python-code.task.ts # Main runtime task implementation and lifecycle.
│ ├── python-code.task-module.ts # Task module registration that references PythonCodeTask.
│ ├── python-code.task-designer.ts # Design-time task metadata and dynamic I/O resolver logic.
│ ├── python-code-settings.component.ts
│ │ # Angular settings component for task configuration.
│ ├── python-code-settings.component.html
│ │ # Settings UI template displayed in Connect IoT designer.
│ └── python-code-settings.component.less
│ # Styles for settings component (currently empty).
│
└── test/
└── unit/
├── tsconfig.json # TypeScript config for unit-test compilation.
└── tasks/
└── python-code/
└── python-code.task.test.ts # Unit test source for PythonCode task behavior.Task Folder Structure
python-code.task.tsdefines runtime behavior.python-code.task-module.tsregisters that task with the engine module system.python-code.task-designer.tsexposes task metadata and input/output behavior at design time.python-code-settings.component.tsprovide the task settings UI used in the designer.python-code-settings.component.htmlprovide the task settings UI.python-code-settings.component.lessprovide the task settings UI css.
The task.ts file will focus on describing the runtime behavior, the other elements are all describing the UI behavior.
Designer
In the task designer we can start by defining what our task inputs and outputs look like. For this task we want to use autoports, inputs and outputs that by dragging and dropping links we can create a connection.
Let’s start by the task designer metadata:
(...)
@TaskDesigner({
name: $localize`:@@@criticalmanufacturing/connect-iot-controller-engine-python-tasks/pythonCode#TITLE:Python Code`,
iconClass: "icon-core-tasks-connect-iot-lg-codeexecution",
inputs: {
autoIn: {
friendlyName: $localize`:@@@criticalmanufacturing/connect-iot-controller-engine-python-tasks/pythonCode#AUTO_INPUT_PORT_TEXT:Link here to add new`,
type: Task.AUTO
},
activate: Task.INPUT_ACTIVATE
},
outputs: {
autoOut: {
friendlyName: $localize`:@@@criticalmanufacturing/connect-iot-controller-engine-python-tasks/pythonCode#AUTO_OUTPUT_PORT_TEXT:Link from here to add new`,
type: Task.AUTO
},
success: Task.OUTPUT_SUCCESS,
error: Task.OUTPUT_ERROR
},
protocol: TaskProtocol.All,
driverOptional: true
})
(...) We start by giving it a better icon and by referencing an autoIn input and an autoOut output. The driverOptional is also important as it will query the user if it wishes to associate the task to a driver, and if so which driver.
We start by creating our settings and inputs and outputs.
export class PythonCodeDesigner implements TaskDesignerInstance, PythonCodeSettings {
// Settings
pyCode: string[];
pyCodeBase64: string;
packages: string[];
contextExpirationInMilliseconds: number;
executionTimeoutMs: number;
// Inputs & Outputs
inputs: Task.TaskInput[];
outputs: Task.TaskOutput[];We also add some code to manage the injection of the autoports into the context. This code is more or less boiler plate, you can check it here.
HTML
In ours task settings we will have static settings and a python code sandbox.
In our html we can start by defining our static settings.
These settings will exist in the Settings tab of the settings page.
As you can see CM already provides a lot of different components we can use, if you are curious about them, feel free to take a look here.
(...) We can also declare our python code sandbox.
(...)