MQTT

Introduction
• Oil and Gas Pipeline data acessing is difficult
• Data can be accessed by Serial Communication, Satellites, Radio
wave communication
• Due to remote location and low bandwidth, difficult to collect the
data on time. If collected also the response is very bad due to
bandwith limitation.
• MQTT developed in 1999 by Andy Stanford Clark and Arlen Nipper
• MQTT is an extremely simple and lightweight messaging protocol.
• Its publish/subscribe architecture is designed to be open and easy
to implement, with up to thousands of remote clients capable of
being supported by a single server.
• These characteristics make MQTT ideal for use in constrained
environments where network bandwidth is low or where there is
high latency and with remote devices that might have limited
processing capabilities and memory
Message Queuing Telemetry
Transport [MQTT]
• Considering the harsh environments in the oil and gas industries, an
extremely simple protocol with only a few options was designed,
with considerations for constrained nodes, unreliable WAN backhaul
communications, and bandwidth constraints with variable latencies.
• These were some of the rationales for the selection of a client/server
and publish/subscribe framework based on the TCP/IP architecture,
Contd..
• An MQTT client can act as a publisher to send data (or resource
information) to an MQTT server acting as an MQTT message broker.
• In Figure the MQTT client on the left side is a temperature (Temp) and
relative humidity (RH) sensor that publishes its Temp/RH data.
• The MQTT server (or message broker) accepts the network connection
along with application messages, such as Temp/RH data, from publishers.
• It also handles the subscription and unsubscription process and pushes
the application data to MQTT clients acting as subscribers.
• The application on the right side of Figure is an MQTT client that is a
subscriber to the Temp/RH data being generated by the publisher or
sensor on the left.
• The presence of a message broker in MQTT decouples the data
transmission between clients acting as publishers and subscribers.
• In fact, publishers and subscribers do not even know (or need to know)
about each other.
• A benefit of having this decoupling is that the MQTT message broker
ensures that information can be buffered and cached in case of network
failures.
Message Queuing Telemetry
Transport [MQTT]
Example
Example
 MQTT Concepts

• Broker
• Client
• Publish/Subscribe Messaging
Models
• Topics
 MQTT Broker
• An MQTT broker is a server that implements the
MQTT protocol. It mediates communication
between MQTT client applications, such as those
running in remote sensors and other devices,
and the enterprise integration layer.
Examples

 WebSphere MQ - began including built-in support for MQTT,

through the WebSphere MQ Telemetry component, starting with
versions 7.0.1 and 7.1.

 Mosquitto is a small, no-cost, open source implementation of an

MQTT broker that supports the MQTT V3.1 protocol. Mosquitto
replicates the functionality of Really Small Message Broker. For
more information about Mosquitto or to download the
Mosquitto broker, refer to this website: http://mosquitto.org/
 MQTT Clients
• An MQTT client (also called a client application) collects
information from a telemetry device, connects to a messaging
server, and uses a topic string to publish the information in a
way that allows other clients or applications to retrieve it.

• An MQTT client also can subscribe to topics, receive

publications associated with those topics, and issue commands
to control the telemetry device.

• Client libraries can simplify the process of writing MQTT client

applications.
 MQTT Publish/Subscribe
messaging model
• MQTT uses a publish/subscribe messaging pattern that enables
a loose coupling between the information provider, called the
publisher, and consumers of the information, called subscribers.
• This is achieved by introducing a message broker between the
publisher and the subscribers.
• Compared with the traditional point-to-point pattern, the
advantage of the publish/subscribe model is that the
publishing device or application does not need to know
anything about the subscribing one, and vice versa.
• The publisher simply sends the message with an identifier that
denotes its topic, or subject area.
• The broker then distributes the message to all applications or
devices that have chosen to subscribe to that topic.
• In this way, the publish/subscribe pattern turns traditional
point-to-point messaging into a multicast of content-based
 MQTT Publish/Subscribe
messaging model
Two examples of publish/subscribe
configurations
Message format
• MQTT is a lightweight protocol because each control packet consists of a 2-byte fixed header with
optional variable header fields and optional payload.

• First MQTT field in the header is Message Type, which identifies the kind of MQTT packet within a
message.
• Fourteen different types of control packets are specified in MQTT.
• Each of them has a unique value that is coded into the Message Type field.

• Next field in the MQTT header is DUP (Duplication Flag).

• This flag, when set, allows the client to note that the packet has been sent previously, but an
acknowledgement was not received.

• QoS header field allows for the selection of three different QoS levels.

• Next field is the Retain flag.

• Only found in a PUBLISH message, the Retain flag notifies the server to hold onto the message
data.
• This allows new subscribers to instantly receive the last known value without having to wait for
the next update from the publisher.

• Last mandatory field in the MQTT message header is Remaining Length.

• This field specifies the number of bytes in the MQTT packet.
Contd..
• MQTT sessions between each client and server consist of four phases:
session establishment, authentication, data exchange, and session
termination.
• Each client connecting to a server has a unique client ID, which allows
the identification of the MQTT session between both parties.
• When the server is delivering an application message to more than
one client, each client is treated independently.
• Subscriptions to resources generate SUBSCRIBE/SUBACK control
packets, while unsubscription is performed through the exchange of
UNSUBSCRIBE/UNSUBACK control packets.
• Graceful termination of a connection is done through a DISCONNECT
control packet, which also offers the capability for a client to
reconnect by re-sending its client ID to resume the operations.
 MQTT methods
MQTT defines methods (sometimes referred to as verbs) to
indicate the desired action to be performed on the identified
resource.

• Connect- Waits for a connection to be established with the

server.
• Disconnect - Waits for the MQTT client to finish any work it
must do, and for the TCP/IP session to disconnect.
• Subscribe- Waits for completion of the Subscribe or
UnSubscribe method.
• UnSubscribe- Requests the server unsubscribe the client
from one or more topics.
• Publish- Returns immediately to the application thread after
passing the request to the MQTT client.
 MQTT methods
MQTT defines methods (sometimes referred to as verbs) to indicate the desired
action to be performed on the identified resource.
• CONNACK - Acknowledges connection request. The CONNACK message is the
message sent by the broker in response to a CONNECT request from a client.

• PINGREQ - PING request. The PINGREQ message is an “are you alive”

message that is sent from, or received by, a connected client.

• PINGRESP - PING response. A PINGRESP message is the response to a

PINGREQ message and means “yes I am alive”.

• PUBACK - Publish acknowledgement. A PUBACK message is the response to

a PUBLISH message with QoS level 1.

• SUBACK - Subscription acknowledgment. A SUBACK message is sent by the

broker to the client to confirm receipt of a SUBSCRIBE message. A SUBACK
message contains a list of granted QoS levels.
 MQ Telemetry Transport
 In MQTT, you use the Last Will and Testament (LWT) feature to
notify other clients about an ungracefully disconnected client.
 Each client can specify its last will message when it connects to a
broker.
 The last will message is a normal MQTT message with a topic,
retained message flag, QoS, and payload.
 The broker stores the message until it detects that the client has
disconnected ungracefully.
 In response to the ungraceful disconnect, the broker sends the
last-will message to all subscribed clients of the last-will message
topic.
 If the client disconnects gracefully with a correct DISCONNECT
message, the broker discards the stored LWT message.
 MQ Telemetry Transport
The MQTT protocol includes the following highlights:
• Open and royalty-free for easy adoption MQTT is open to make
it easy to adopt and adapt for the wide variety of devices,
platforms, and operating systems that are used at the edge of a
network.
• A publish/subscribe messaging model that facilitates one-to-
many distribution Sending applications or devices do not need
to know anything about the receiver, not even its address.
• Ideal for constrained networks (low bandwidth, high latency,
data limits, and fragile connections) MQTT message headers
are kept as small as possible. The fixed header is just two bytes,
and its on demand, push-style message distribution keeps
network utilization low.
• Multiple service levels allow flexibility in handling different
types of messages Developers can designate that messages will
be delivered at most once, at least once, or exactly once.
 MQ Telemetry Transport
• Designed specifically for remote devices with little
memory or processing power Minimal headers, a small
client footprint, and limited reliance on libraries make
MQTT ideal for constrained devices.
• Easy to use and implement with a simple set of command
messages Many applications of MQTT can be
accomplished using just CONNECT, PUBLISH, SUBSCRIBE,
and DISCONNECT.
• Built-in support for loss of contact between client and
server The server is informed when a client connection
breaks abnormally, allowing the message to be re-sent or
preserved for later delivery.
 MQTT client application that can publish and
subscribe

The following general steps are performed when creating

the applications:
1. Create an instance of an MQTT client.

2. Prepare connection options and connect to the MQTT

broker using these options.

3. Publish messages to topics, and subscribe to topics.

4. Disconnect from the MQTT broker (and remember to

release resources on platforms without automatic memory
management).
import machine
import network
import time
import dht

from umqtt.simple import MQTTClient

d = dht.DHT11(machine.Pin(4))

print(" Step 1 import complete")

#
# connect the ESP8266 to local wifi network
#

sta_if= network.WLAN(network.STA_IF)
if not sta_if.isconnected():
print('connecting to network...')
sta_if.active(True)
sta_if.connect('IoTLab', ‘password') WiFi Connection
while not sta_if.isconnected():
pass

print(" Step 2 network comnnected complete")

print('network config:', sta_if.ifconfig())
#
# connect ESP8266 to Thingspeak using MQTT
#
myMqttClient = "My-IOT-Test" # can be anything unique
thingspeakIoUrl = "mqtt.thingspeak.com"
c = MQTTClient(myMqttClient, thingspeakIoUrl, 1883) # uses unsecure TCP connection Client Connection –
c.connect()
print(" Step 3 C:", c ) Thingspeak.com
 Thingspeak.com channel
credentials

#
# publish temperature and free heap to Thingspeak using MQTT
#

thingspeakChannelId = "336551" #"YOUR-CHANNEL-ID" # <--- replace with your Thingspeak Channel ID

thingspeakChannelWriteapi = "LSH8DNKECU64X0DN" #"YOUR-CHANNEL-WRITEAPIKEY" # <--- replace with
your Thingspeak Write API Key
publishPeriodInSec = 2
while True:
d.measure()
tempInDegC = d.temperature()
hum=d.humidity()
print('Hello world! Measuring Temp and Hum')
print("Todays Temp is", d.temperature(), d.humidity())
Data Publish to channe
# note: string concatenations below follow best practices as described in micropython reference doc
credentials = "channels/{:s}/publish/{:s}".format(thingspeakChannelId, thingspeakChannelWriteapi)
payload = "field1={:.1f}&field2={:d}\n".format(d.temperature(), gc.mem_free())
c.publish(credentials, payload)

time.sleep(publishPeriodInSec)

c.disconnect()
MQTT Quality of Service (QoS)
• Quality of service (QoS) levels should be understood by every user
of MQTT.
• A QoS value determines how each message will be delivered and
must be specified for every message sent by means of MQTT
 MQTT Quality of Service (QoS)
• Quality of service (QoS) levels should be understood by every user of
MQTT.
• A QoS value determines how each message will be delivered and must
be specified for every message sent by means of MQTT
MQTT Quality of Service (QoS)
• Quality of service (QoS) - 0 , At most once delivery
• messages are delivered according to the best effort of the
underlying network.
• A response is not expected and no retry semantics are
defined in the protocol.
• The message arrives at the broker either once or not at all.
This is the least level of QoS.
• The MQTT command message used is PUBLISH.
• No other command messages flow for the QoS 0 messages.
 MQTT Quality of Service (QoS)
• Quality of service (QoS) - 1 , At least Once Delivery
• message is delivered at least once.
• The MQTT client or the server would attempt to deliver the
message at least once, but there can be a duplicate message.
• The receipt of a message by the broker is acknowledged by a
PUBACK message. If there is an identified failure of either the
communications link or the sending device, or the
acknowledgment message is not received after a specified
period of time, the sender resends the message with the DUP
bit set in the message header.
• The message arrives at the broker at least once. Both
SUBSCRIBE and UNSUBSCRIBE messages use QoS level 1.
• If the client does not receive a PUBACK message (either within
a time period defined in the application, or if a failure is
detected and the communications session is restarted), the
client resends the PUBLISH message with the DUP flag set
MQTT Quality of Service (QoS)
 MQTT Quality of Service (QoS)
• Quality of service (QoS) - 2 , Exactly once delivery
• highest level of QoS.
• Additional protocol flows above QoS level 1 ensure that
duplicate messages are not delivered to the receiving
application.
• The message is delivered once and only once when QoS 2 is
used. The MQTT client or the server will ensure that the
message is sent only once.
• This QoS must be used only when duplicate messages are not
desired. From a performance perspective, there is a price to
be paid in terms of network traffic and processing power.
• The MQTT command messages used are PUBLISH, PUBREC
(publish received), PUBREL (publish release) and PUBCOMP
(publish complete)
• The message is sent in the PUBLISH flow and the client will
MQTT Quality of Service (QoS)
• Quality of service (QoS) - 2 , Exactly once
delivery
MQTT Vs HTTP