Solution Overview

DTU Introduction

DTU (Data Transfer Unit) is a wireless terminal device specially used to convert serial port data into network data or convert network data into serial port data for wireless communication through networks. It has many advantages such as flexible networking, wide network coverage, short construction cycle, and low cost. It is widely used in meteorology, hydrology, geology and other industries.

Basic Principles

The main function of DTU is to transmit the device data wirelessly back to the cloud (backend service), as shown in Figure 1: DTU Introduction. To complete the data transmission, a complete data transmission system needs to be established. This system includes: DTU device, customer device (MCU), mobile network, backend center (cloud). Insert a SIM card into the DTU device. After the device is powered on, it will first dial and register the network; then establish a connection with the cloud center service through the network; After that, the DTU device can communicate bidirectionally with the cloud backend service —— receive MCU data through the serial port, process it and report it to the cloud (uplink data), and process the data received from the cloud and send it to the MCU through the serial port (downlink data).

Figure 1: DTU Introduction > The sensor (temperature, humidity sensor, etc.) collects data and sends it to the MCU device end. The MCU device end sends the collected data to the cloud end server through the serial port of DTU. After receiving the data, the server can analyze, process, display, save, etc.

Application Areas

Figure 2: DTU Application Areas ## Hardware Selection

This solution supports multiple QuecPython hardware devices. The following introduces the QuecPython EVB development board and DP-DTU-Q600.

QuecPython_EVB Development Board Introduction

Take the QuecPython_EC2X_EVB_V2.0 development board (EC2X development board introduction: https://python.quectel.com/doc/Getting_started/en/evb/ec2x-evb.html) equipped with the EC200U module as an example. As shown in the figure below, the device supplies power to the module through type-c, and connects UART to the PC via the TTL to USB module for easy debugging.

Figure 3: QuecPython_EC2X_EVB_V2.0 connects to PC via UART and TTL to USB

Pin on development board	TTL to USB module	Color of wire in picture
Pin 13 (TX) of J7	RX	Red
Pin 14 (RX) of J7	TX	Orange
Pin 3 (GND) of J7	GND	Yellow

For more development boards, please refer to: https://python.quectel.com/doc/Getting_started/en/hardware_prepare.html.

DP-DTU-Q600 Product Introduction

DP-DTU-Q600 is a DTU with LTE CAT1 network, which is convenient to be integrated into industrial computers, industrial equipment, sensing devices, etc. Its physical size is 72mm x 40.5mm x 20mm. The product has the characteristics of wide network coverage and low transmission latency. It supports access of three major operators; The product has three indicator lights for customers to easily confirm the device status.

Figure 4: DP-DTU-Q600 Product Specifications:

Specification	Parameter
Power supply range	DC5-24V
DC5-24V	2PIN terminal
Antenna interface	SMA external thread holes
SIM card interface	Pop-up NANO SIM card slot
Network system	4G CAT1 full Netcom
Communication interface	2PIN terminal RS485

Product Features:

• Built-in TCP/IP protocol stack, and embedded operating system, with dial-up Internet access and TCP/IP data communication functions
• Provides serial data bidirectional conversion function
• Support heartbeat to keep always online
• Support parameter configuration, permanently saved
• Support user serial port parameter settings
• Support user secondary development
• IoT card, no special card required, general IoT cards can be used
• NANO SIM card slot
• RS485 interface for easy integration

Device Debugging:

• Insert the SIM card into the NANO SIM card slot
• Connect the antenna
• Connect the power supply

Figure 5: DP-DTU-Q600 Hardware Interface Connect the device to the development machine (this article uses `CP2102 USB to TTL module` for connection and debugging).

Use 2 DuPont lines to connect the 485A and 485B pins, and connect the CP2102 to the USB port of the development machine (as shown in Figure 6).

Figure 6: Connect DP600R to development machine > After successful connection, when the device is powered on and connected to the network, the `power` and `net` indicator lights will light up.

Debug with QPYcom

Refer to the document for installing QPYCom and building the QuecPython development environment: https://python.quectel.com/doc/Application_guide/en/dev-tools/QPYcom/index.html.

Open repl serial port

Figure 7: QPYCom open repl serial port ##### Download Script

Create download project - Switch to the Download tab, click Create Project, and enter any project name.

Here we create a project named dtu.

Figure 8: QPYCom create new project Select import file - Right-click the `usr` directory, click **Import All** in the pop-up dialog box, and then select the **code** folder in our DTU code repository - import all code in code into the module in one click. As shown below:

Figure 9: QPYCom one-click import script Import script - Click the `Download Script` button in the lower right corner to start importing the script.

Run DTU Application

Switch to the "Files" tab, select "dtu.py" on the right, and click the Run button to start dtu debugging and running. To run automatically after power-on, just rename "dtu.py" to "main.py".

Figure 10: QPYCom run script DTU runs successfully. In the "REPL" window of QPYcom, you can observe the following printout.

The application relies on specific user parameter configurations, such as configuring MQTT connection parameters. Refer to the subsequent DTU scheme introduction section.

Figure 11: QPYCom script output log ## Software Design of the Solution

System Diagram

Figure 12: DTU Solution System Diagram Component Description:

• Various cloud objects: Inherit modules.CloudObservable and override corresponding methods, mainly to implement cloud initialization, connection and data sending and receiving functions.
• Subscriber: Inherit modules.CloudObserver and override corresponding methods, mainly to subscribe and receive message notifications from cloud objects, and call different executors according to message types for processing.
• Executor: Mainly used for specific business processing, called by the subscriber. The default includes downlink executor (processing downlink data forwarding serial port), OTA upgrade executor (processing upgrade messages and triggering upgrades), and uplink executor (mainly used to process uplink data).
• Publisher: Mainly associated with cloud objects, used to publish data to the cloud.

Component Collaboration Process:

• Build serial port object - At least one serial port is usually used as a communication channel, and multiple or one serial port is selected according to the actual situation.
• Build downlink data executor and OTA upgrade executor - The executor is mainly used for business processing. Users need to implement specific business processing logic methods in the executor.
  • The downlink data executor needs to add a serial port object to process downlink messages - forwarded through the serial port.
  • OTA upgrade executor needs to implement specific upgrade business process.
• Create cloud object - Support Alibaba Cloud, Tencent Cloud, MQTT, etc., import and use according to actual business needs.
• Create subscriber.
  • The cloud object adds the subscriber as its observer.
  • Subscribers can have multiple, corresponding to multi-channel downlink message processing.
• Create publisher.
  • Add cloud object as the target object for the publisher to publish messages.
• Create uplink data executor.
  • The uplink data executor is mainly used to receive serial port data processing.
  • Add a publisher for the executor to publish messages. How the message is published is determined by the publisher.

Downlink data supports multiple channels. The subscriber can have multiple as an observer. Once a message is downlinked after a cloud adds multiple subscribers, it will notify all subscribers. Different subscribers may handle the data through different downlink executors or OTA upgrade executors.

Uplink data supports multiple channels. When multiple serial port data need to be monitored, there can be multiple uplink executors to forward multiple serial port data to the cloud.

Observer Pattern

• Concept: When there is a one-to-many relationship between objects, the observer pattern (Observer Pattern) is used. For example, when an object is modified, it will automatically notify dependent objects. The observer pattern belongs to the behavioral pattern.

• Intent: Define a one-to-many dependency relationship between objects. When the state of an object changes, all dependent objects are notified and updated automatically.

• Mainly solve: The problem of notifying other objects when the state of an object changes, while considering ease of use and loose coupling to ensure a high degree of collaboration.

• When to use: When the state of an object (target object) changes, all dependent objects (observer objects) will be notified to broadcast notification.

• Key Code: There is a list in the abstract class to hold observers.

In the design of the DTU solution, a cloud class object (Socket and MqttIot) is an observable, which will notify RemoteSubscribe (subscriber, an observer) when there is downlink data. , and RemoteSubscribe will call different executors for processing according to different businesses.

Component Class Design and Interaction

Class Inheritance

Cloud Class Inheritance Diagram

CloudObservable: Observable class, mainly used to inherit and override when defining various cloud object classes. The main user interfaces are:

• init: cloud initialization

• addObserver: Add an observer object

• delObserver: Remove an observer object

• notifyObservers: Notify all observers

• through_post_data: transparent data publishing

• post_data: cloud message publishing

• ota_request: publish OTA upgrade request on cloud

• ota_action: execute upgrade action

Subscriber Class Inheritance Diagram

CloudObserver: Observer object class, mainly used for passive notification reception. The main methods are:

• execute: This method will be called to perform specific actions after being notified by the observable.

RemoteSubscribe: Subscriber, is an observer, used to receive cloud downlink data notifications. The main methods are:

• execute: When notified, this method is called to perform specific business processing procedures. (Use different executors to handle different businesses, define 2 types of executors in this class object: __executor (ordinary executor), __ota_executor (dedicated to executing OTA upgrade executor))
• add_executor: Add executor.

Downlink Data Processing

Downlink data processing sequence diagram:

This solution defines two types of executors for processing downlink data:

• DownlinkTransaction: Downlink data executor, responsible for forwarding cloud data through serial port.
• OtaTransaction: OTA upgrade executor, responsible for processing upgrade related operations.

For example, when there is downlink data, if it is transparent data, the DownlinkTransaction executor will be called to forward the data to the MCU through the serial port; if it is an OTA upgrade message, the OtaTransaction executor will be called. Perform upgrade operations.

Uplink Data Processing

Uplink data processing sequence diagram:

This solution defines one type of uplink data executor:

• UplinkTransaction: Uplink data executor, used to receive serial port data and upload it to the cloud.
  • Main attributes:
    • __remote_pub: Remote publisher, used to upload data to the cloud. It is an object of class modules.remote.RemotePublish.
    • __serial: Serial port object, used for serial port data sending and receiving. Defined in modules.serial.Serial.
  • Main methods:
    • uplink_main: Used to receive serial port data and upload it to the cloud.

User Guide

Write Configuration File

DTU configuration file path: code/dtu_config.json.

Make the following configurations based on private mqtt cloud:

• System configuration

• Private mqtt cloud configuration

Device interface call flow (interface call logic, parameter configuration), upper computer tool configuration parameters, cloud configuration

DTU Initialization Process and Interface Description

The application class Dtu is defined in code/dtu.py. Its instance is the application object. Calling the start method starts the application service.

from usr.dtu import Dtu

if __name__ == "__main__":
    dtu = Dtu()
    dtu.start()

Dtu Class Interface Description

The following methods are defined in the Dtu class, as described below (see comments in code for full description, complete code refer to dtu.py usage example):

class Dtu(Singleton):

    def __init__(self):
        # Define a timer to periodically check OTA upgrade plan
        self.__ota_timer = osTimer()  
        # OTA upgrade executor, used to perform OTA upgrade related operations
        self.__ota_transaction = None

    def __cloud_init(self, protocol):
        # Initialize cloud object and connect to cloud according to protocol
        # This parameter corresponds to config item `system_config.cloud`
        pass

    def __periodic_ota_check(self, args):
        # Periodically check OTA upgrade plan
        pass

    def start(self):
        # Initialize DTU and start services
        pass

Application Service Initialization Process Description

After the start function is called, the DTU application service is initialized and various services are started.

Initialization process:

• Load system configuration

• Create serial communication object (usr.modules.serial.Serial)

• Create cloud object (usr.modules.mqttIot.MqttIot or other cloud objects)

• Create GUI tool communication object (usr.modules.dtu_transaction.GuiToolsInteraction)

• Create uplink data executor (usr.modules.dtu_transaction.UplinkTransaction)

• Create downlink data executor (usr.modules.dtu_transaction.DownlinkTransaction)

• Create OTA upgrade executor (usr.modules.dtu_transaction.OtaTransaction)

• Create subscriber (usr.modules.remote.RemoteSubscribe)

• Create publisher (usr.modules.remote.RemotePublish)

• Start timer to periodically check OTA upgrade plan

• Create service thread to continuously read serial port data, parse and upload to cloud

Detailed description of process steps:

(1) Load system configuration

from usr.settings import settings

settings is a global configuration file (Settings) object, corresponding to the /usr/dtu_config.json configuration file parameters, persisted in json format.

Methods:

• get: Get all current configuration parameters (i.e. dict type data imported from json file)
• set(opt, val): Set configuration item opt to parameter val.
• save: Persistently save configuration.
• remove: Delete configuration file.
• reset: Restore default configuration.
• set_multi(**kwargs): Set parameters in batches.

(2) Create serial communication object

from usr.modules.serial import Serial

# Serial initialization
serial = Serial(int(uart_setting.get("port")),
                int(uart_setting.get("baudrate")),
                int(uart_setting.get("databits")),
                int(uart_setting.get("parity")),
                int(uart_setting.get("stopbits")),
                int(uart_setting.get("flowctl")),
                uart_setting.get("rs485_direction_pin"))

serial is a serial communication object (Serial).

Methods:

• write(data): Pass in written data bytes.
• read(n, timeout): Read n bytes from serial port, timeout is timeout.

(3) Create cloud object

The Dtu.__init_cloud method is used to create a cloud object (create different cloud objects according to the system configuration file).

class Dtu(object):

    def __init_cloud(self, protocol):
        if protocol == 'mqtt':
            cloud_config = settings.current_settings.get("mqtt_private_cloud_config")
            client_id = cloud_config["client_id"] if cloud_config.get("client_id") else modem.getDevImei()
            cloud = MqttIot(cloud_config.get("server", None),
                                int(cloud_config.get("qos", 0)),
                                int(cloud_config.get("port", 1883)),
                                cloud_config.get("clean_session"),
                                client_id,
                                cloud_config.get("username"),
                                cloud_config.get("password"),
                                cloud_config.get("publish"),
                                cloud_config.get("subscribe"),
                                cloud_config.get("keep_alive")
                                )
            cloud.init(enforce=True)
            return cloud
        else:
            # Omitted, refer to `code/dtu.py` for other cloud object initialization
            pass

    def start(self):
        # ...
        cloud = self.__cloud_init(settings.current_settings["system_config"]["cloud"])
        # ...

Take MQTT as an example, read mqtt configuration parameters from the system configuration file, instantiate the MqttIot object with passed parameters, and call the MqttIot.init method to complete the cloud object initialization operation.

Methods:

• addObserver: Add observer object, used by the cloud to handle it after receiving downlink data.
  • Subscriber: When receiving cloud notification (carrying downlink data), call executor to process.

(4) Create GUI tool communication object

gui_tool_inter = GuiToolsInteraction()

GUI communication object GuiToolsInteraction, used to parse upper computer instructions issued from serial port, and process specific instruction operations.

Methods:

• parse_serial_data: Parse serial data.

(5) Create uplink data executor

up_transaction = UplinkTransaction()
up_transaction.add_module(serial)
up_transaction.add_module(gui_tool_inter) 

Methods:

• add_module: Add module object.
  • Serial port object: Uplink executor reads serial port data through serial port object.
  • GUI communication object, parse the read serial port data through GUI object.
  • Publisher object, publish data to cloud through this object.
• uplink_main: Read serial port data, and use GUI object to parse (if it is instruction data, process instruction operation, if it is uplink data, forward it to cloud).

(6) Create downlink data executor

down_transaction = DownlinkTransaction()
down_transaction.add_module(serial)

Methods:

• add_module: Add module object
  • Serial port object: Downlink executor forwards downlink data through serial port object.
• downlink_main: Used to process downlink data.

(7) Create OTA upgrade executor

ota_transaction = OtaTransaction() 

Methods:

• ota_check: Check OTA upgrade plan
• event_ota_plain: Respond to upgrade plan (verify parameters and start upgrade process after receiving issued upgrade plan)
• add_module: Add module object
  • Publisher object: Report upgrade status or actively request upgrade plan through the publisher object.

(8) Create subscriber

remote_sub = RemoteSubscribe()
remote_sub.add_executor(down_transaction, 1)
remote_sub.add_executor(ota_transaction, 2)
cloud.addObserver(remote_sub)

Methods:

• add_executor: Add executor.
  • Downlink executor: Forward data through serial port.
  • OTA upgrade executor: Parse downlink upgrade message and process upgrade request.

(9) Create publisher

remote_pub = RemotePublish()
remote_pub.add_cloud(cloud)
up_transaction.add_module(remote_pub)
ota_transaction.add_module(remote_pub) 

Methods:

• add_cloud: Add cloud object, use this cloud object to publish messages.

(10) Start timer to periodically check OTA upgrade plan

# Check upgrade plan every 600 seconds
self.__ota_timer.start(1000 * 600, 1, self.__periodic_ota_check) 

(11) Create service thread to continuously read serial port data, parse and upload to cloud

# Start uplink transaction
_thread.start_new_thread(up_transaction.uplink_main, ())

Start a separate thread to execute up_transaction.uplink_main. The uplink_main method continuously reads serial port data, uses GuiToolsInteraction to parse serial port data to process upper computer instructions, or uses RemotePublish to forward data to the cloud.

Product Solution

Smart Agriculture Central Control Panel