Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Datrust-Project/trustworthy-digital-twin

BranchesTags

Repository files navigation

Trustworthy Digital Twin - Java

The Trustworthy Digital Twin (TDT) is a Java-based demonstrator and prototype focusing on exploring and evaluating the concept of data flow trustworthiness. The TDT acts as the digital counterpart of a physical device that provides telemetry data (e.g., from an accelerometer), integrating functionalities for monitoring, dynamic configuration, and performance metrics computation.

The main goal of this repository is to provide a practical reference for the design, implementation, and observation of distributed Digital Twins capable of:

  • reliably collecting and managing IoT data
  • performing local computations to emulate complex workloads
  • exposing metrics and state information through standard digital interfaces such as HTTP, Prometheus, and MQTT.

The DT has the following characteristics:

  • MQTT Physical Adapter to read data from a physical device providing accelerometer telemetry data
    • Source Topic: "device/<device_id>/sensor/sensor_accel/telemetry"
    • Expected Payload: {"id": "sensor_accel", "value": {"accel_x": 0.04, "accel_y": 0.05, "accel_z": 0.06}, "last_update_time_ms": 1726825824293}
  • The Shadowing Function:
    • build the DT State with accelerometer telemetry data
    • Integrate the computation of Prime Number to emulate a complex computation task
  • The DT compute ODTE Metrics according to the provided configuration parameters mainly related to:
    • Expected Message Rate
    • Prime Numbers Computation Count
    • ODTE Configuration Parameters
  • HTTP Digital Adapter to expose the DT State (Polling)
  • The DT has additional modules:
    • An HTTP Prometheus Adapter to expose the collected metrics
    • A Custom HTTP API to read local collected metrics
    • A Configuration Management API to read and update the DT configuration

Preconditions

Required deployment preconditions are the following:

  • Execute Physical MQTT Broker (e.g., port 1883) associated to data coming from physical device
  • (Optional if we add MQTT Digital Adapter) Execute Digital MQTT Broker (e.g., port 1884) associated to data coming from digital twins

Trustworthy DT (Docker Container)

In the folder docker it is possible to find the scripts to run a Trustworthy DT that associated to a physical device generating data associated to accelerometer sensors on the topic.

In the folder single we have all the resources to run a single DT associated to a single physical device generating data:

  • dt_conf.yaml: Configuration file for the DT with the following core parameters:
    • digitalTwinId: Id of the Digital Twin
    • targetDeviceId: Id of the Physical Device used also in the MQTT Topic to read data
    • physicalMqttBrokerAddress: IP Address of the Physical MQTT Broker
    • physicalMqttBrokerPort: Port of the Physical MQTT Broker

Exposed Ports:

  • 5555: HTTP API to read collected metrics + Configuration Management API
  • 3000: HTTP Digital Adapter to expose the DT State
  • 1234: HTTP Prometheus Module to expose the collected metrics

API - PostMan Collection

In the folder api it is possible to find a PostMan Collection that can be used to interact with the HTTP API exposed by the DTs.

Grafana Dashboard Configuratio(s)

In the folder grafana it is possible to find the JSON configuration file to import the Grafana Dashboard used to monitor the DTs. Available Dashboards:

  • DT: To monitor the collected metrics
  • Composed DTs: To monitor the collected metrics of composed DTs

Run Single DT (No Container)

In order to run the example with a single emulated physical device and one associate DT follow these steps:

  • Run Java Class it.unimore.dipi.iot.smartobject.process.SensorSmartObjectProcess emulating physical device telemetry data
  • Run Java Class it.unimore.dipi.iot.consumer.DeviceMQTTConsumer to read MQTT packets generated by the physical device
  • Properly configure file dt_conf.yaml with the right IP Addresses and Port for active MQTT Brokers (physical and digital) that will be used by the DT to properly communicate
  • Run Java Class it.unimore.dipi.iot.digitaltwin.MqttDigitalTwinProcess to execute the DT associated to the physical device
  • Run Java Class it.unimore.dipi.iot.consumer.DigitalTwinMQTTConsumer to read MQTT packets generated by the DT
  • Check how original physical device data use a custom JSON Payload while the DT transform it into a standard SenML data format

Each DT (if the configuration flag is enabled) can generate local metrics data associated to communication and processing tasks. The DT exposes a dedicated HTTP API allowing an external client to read the whole list of collected metrics. The API is exposed on the following base path: http://<dt_ip_address>:<dt_metric_port> (e.g., http://127.0.0.1:5555). Modeled endpoints are:

Metric Folders List

Url: http://<dt_ip_address>:<dt_metric_port>/metrics

Example Url: http://127.0.0.1:5555/metrics

Response:

[
    "1646144499282",
    "1646154123046"
]

Metric List

Url: http://<dt_ip_address>:<dt_metric_port>/metrics/<metric_folder>

Example Url: http://127.0.0.1:5555/metrics/1646144499282

Response:

[
  "it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.mqtt_topics_registration_time.csv",
  "mqtt_pp_senml.it.unimore.dipi.things:wldt:00dd17c4-32dc-4e97-9508-642ae3707443.execution_time.csv",
  "it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.mqtt_outgoing_publish_time.csv",
  "it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.incoming_mqtt_payload_size.csv",
  "it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.incoming_telemetry_mqtt_payload_size.csv",
  "it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.incoming_mqtt_client_setup_time.csv",
  "it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.outgoing_mqtt_client_setup_time.csv",
  "it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.mqtt_forward_time.csv"
]

Single Metric Details

Url: http://<dt_ip_address>:<dt_metric_port>/metrics/<metrics_file>

Example Url: http://127.0.0.1:5555/metrics/1646144499282/it.unimore.dipi.iot.wldt.worker.mqtt.Mqtt2MqttWorker.mqtt_forward_time.csv

Response:

[
  {
    "t": "1646144509",
    "count": "11",
    "max": "84.127903",
    "mean": "10.302977",
    "min": "1.169330",
    "stddev": "22.657384",
    "p50": "2.319931",
    "p75": "7.150386",
    "p95": "84.127903",
    "p98": "84.127903",
    "p99": "84.127903",
    "p999": "84.127903",
    "mean_rate": "1.294489",
    "m1_rate": "1.200000",
    "m5_rate": "1.200000",
    "m15_rate": "1.200000",
    "rate_unit": "calls/second",
    "duration_unit": "milliseconds"
  },
  {
    "t": "1646144519",
    "count": "23",
    "max": "84.127903",
    "mean": "5.807295",
    "min": "1.154028",
    "stddev": "15.605856",
    "p50": "2.450505",
    "p75": "3.054931",
    "p95": "9.313971",
    "p98": "84.127903",
    "p99": "84.127903",
    "p999": "84.127903",
    "mean_rate": "1.243590",
    "m1_rate": "1.200000",
    "m5_rate": "1.200000",
    "m15_rate": "1.200000",
    "rate_unit": "calls/second",
    "duration_unit": "milliseconds"
  },
  [..]
]

Run Single DT (With Docker Container)

Only the first time (or if you change the DT Codebase) we have to build the DT container using the following command:

./docker_build.sh 0.1

To check if the container has been built and it is available you can run:

docker images

and you should see the following record:

REPOSITORY                                                                                    TAG  IMAGE ID       CREATED          SIZE
registry.gitlab.com/piconem-university/projects/dt-software-patterns/wldt-digital-twin-mqtt   0.1  c6eba7dee7d9   40 minutes ago   705MB

In order to run the example with a single emulated physical device and one associate DT follow these steps:

  • Run Java Class it.unimore.dipi.iot.smartobject.process.SensorSmartObjectProcess emulating physical device telemetry data
  • Run Java Class it.unimore.dipi.iot.consumer.DeviceMQTTConsumer to read MQTT packets generated by the physical device
  • Move to folder docker/single
  • Run the script start_wldt_container.sh
  • Run Java Class it.unimore.dipi.iot.consumer.DigitalTwinMQTTConsumer to read MQTT packets generated by the DT
  • Check how original physical device data use a custom JSON Payload while the DT transform it into a standard SenML data format
  • You can stop the DT with the script stop_wldt_container.sh

Run Multiple DT (With Docker Containers)

In order to run the example with a 3 emulated physical device and 3 associate DTs follow these steps:

  • Run Java Class it.unimore.dipi.iot.smartobject.process.DeploymentProcess emulating 3 physical devices telemetry data
  • Run Java Class it.unimore.dipi.iot.consumer.DeviceMQTTConsumer to read MQTT packets generated by the physical devices
  • Move to folder docker/multiple
  • Run the script start_dt_deployment.sh
  • Run Java Class it.unimore.dipi.iot.consumer.DigitalTwinMQTTConsumer to read MQTT packets generated by active DTs
  • Check how original physical device data use a custom JSON Payload while the DT transform it into a standard SenML data format
  • You can stop all the active DTs with the script stop_dt_deployment.sh

HTTP API - Configuration Management

A dedicated HTTP API has been added to read the current device configuration and control parameters related to sensor update period (ms) and update initial delay.

Configuration Resource URL: http://<ip_address>:<server_port>/conf

READ The Current Configuration

HTTP METHOD: GET URL: http://<ip_address>:<server_port>/conf REQUEST BODY: Empty RESPONSE CODE: 200 OK RESPONSE BODY:

{
  "digitalTwinId": "testDT",
  "targetDeviceId": "testDevice1",
  "physicalMqttBrokerAddress": "127.0.0.1",
  "physicalMqttBrokerPort": 1883,
  "digitalMqttBrokerAddress": "127.0.0.1",
  "digitalMqttBrokerPort": 1884,
  "metricsEnabled": true,
  "httpApiPort": 5556,
  "prometheusHttpApiPort": 1234,
  "expectedMsgSec": 1.0,
  "primeNumbersComputationCount": 10000,
  "observationBucketStart": 0.0,
  "observationBucketWidth": 0.005,
  "observationBucketCount": 50
}

UPDATE The Current Configuration (Option 1 - Full Conf)

HTTP METHOD: PUT URL: http://<ip_address>:<server_port>/conf REQUEST BODY:

{
  "digitalTwinId": "testDT",
  "targetDeviceId": "testDevice1",
  "physicalMqttBrokerAddress": "127.0.0.1",
  "physicalMqttBrokerPort": 1883,
  "digitalMqttBrokerAddress": "127.0.0.1",
  "digitalMqttBrokerPort": 1884,
  "metricsEnabled": true,
  "httpApiPort": 5556,
  "prometheusHttpApiPort": 1234,
  "expectedMsgSec": 1.0,
  "primeNumbersComputationCount": 10000,
  "observationBucketStart": 0.0,
  "observationBucketWidth": 0.005,
  "observationBucketCount": 50
}

RESPONSE CODE: 200 OK RESPONSE BODY: Empty

UPDATE The Current Configuration (Option 2 - Only Some Parameters)

HTTP METHOD: PUT URL: http://<ip_address>:<server_port>/conf REQUEST BODY:

{
  "expectedMsgSec": 10,
  "primeNumbersComputationCount": 20000
}

RESPONSE CODE: 200 OK RESPONSE BODY: Empty

About

Trustworthy Digital Twin

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages