Optimizing High Memory Usage in Edge IoT Manager: Scaling 50+ Resource

Integrations
,
1

I am currently working on a critical manufacturing customization project using an IoT Automation Manager designed for edge devices. While the architecture is functional for small-scale deployments, I am encountering significant scaling issues regarding memory consumption.

The Setup
My current test environment consists of:
• 1 Automation Manager (Orchestration layer)
• 50 Controllers
• 50 Drivers

The Problem
This architecture results in approximately 110 individual Node.js processes. Currently, this setup consumes ~20 GB of RAM, which is prohibitively expensive for standard edge gateway hardware.
While the manager is marketed as “lightweight,” the overhead of the Node.js runtime for each individual node process (Manager + Controller + Driver) is causing a massive memory footprint that doesn’t scale linearly with the hardware typically available at the edge.

Questions for the Community:

  1. *Process Consolidation: *Are there proven patterns for consolidation? Specifically, is it feasible to run multiple Drivers/Controllers within a single worker thread or process to reduce the V8 engine overhead?
  2. Resource Management: For those using Node-based edge managers, how do you handle the baseline memory footprint per resource? Are there specific flags (e.g., --max-old-space-size or --gc-interval) you’ve found effective for dense deployments?
    3.* Alternative Architectures: *Given the requirement for 50+ resources, should I be looking into a more modular “Plugin” architecture rather than a “Process-per-Resource” model?
  3. *Shared Runtimes: *Are there ways to share the Node runtime across these 110 processes to avoid 110 separate heaps?
2

Hi,

I am curious, do you have one OPC-UA server instance per machine? If not, what I’ve seen done is map the controller instance to a Resource of type Component and then have all the Resources be parents of this Resource Component. This Controller and Driver, in runtime can now resolve all the parent resources and use a single controller-driver for all those resources. Then when ingesting the events, you retrieve the expected Resource related to that event and can do whatever logic you are doing. Of course, it may be that just with deconstructing the tag you already now the expected Resource, which means that you won’t need to know what are the parent resources, just deconstruct the tags and know everything you need.

There are also, other approaches here https://devblog.criticalmanufacturing.com/blog/20230413_iot_mes_as_configuration_owner.

Bear in mind that this is all about a trade off between footprint and resiliency. There are many other approaches, the one-to-one is the one that maximizes resiliency but there are many other. Even the one I suggested, can be more or less resilient depending the amount of parent resources you add to the Component Resource.