AWS IoT Device SDK for Python

The AWS IoT Device SDK for Python allows developers to write Python script to use their devices to access the AWS IoT platform through MQTT or MQTT over the WebSocket protocol. By connecting their devices to AWS IoT, users can securely work with the message broker, rules, and the device shadow (sometimes referred to as a thing shadow) provided by AWS IoT and with other AWS services like AWS Lambda, Amazon Kinesis, Amazon S3, and more.

Overview
Installation
Use the SDK
Key Features
Examples
API Documentation
License
Support

Overview

This document provides instructions for installing and configuring the AWS IoT Device SDK for Python. It includes examples demonstrating the use of the SDK APIs.

MQTT Connections

The SDK is built on top of a modified Paho MQTT Python client library. Developers can choose from two types of connections to connect to AWS IoT:

MQTT (over TLS 1.2) with X.509 certificate-based mutual authentication.
MQTT over the WebSocket protocol with AWS Signature Version 4 authentication.

For MQTT over TLS (port 8883), a valid certificate and a private key are required for authentication. For MQTT over the WebSocket protocol (port 443), a valid AWS Identity and Access Management (IAM) access key ID and secret access key pair are required for authentication.

Device Shadow

A device shadow, or thing shadow, is a JSON document that is used to store and retrieve current state information for a thing (device, app, and so on). A shadow can be created and maintained for each thing or device so that its state can be get and set regardless of whether the thing or device is connected to the Internet. The SDK implements the protocol for applications to retrieve, update, and delete shadow documents. The SDK allows operations on shadow documents of single or multiple shadow instances in one MQTT connection. The SDK also allows the use of the same connection for shadow operations and non-shadow, simple MQTT operations.

Installation

Minimum Requirements

Python 2.7+ or Python 3.3+
OpenSSL version 1.0.1+ (TLS version 1.2) compiled with the Python executable for X.509 certificate-based mutual authentication

To check your version of OpenSSL, use the following command in a Python interpreter:
```
>>> import ssl
>>> ssl.OPENSSL_VERSION
```

Install from pip

pip install AWSIoTPythonSDK

Build from source

git clone https://github.com/aws/aws-iot-device-sdk-python.git
cd aws-iot-device-sdk-python
python setup.py install

Download the zip file

The SDK zip file is available here. Unzip the package and install the SDK like this:

python setup.py install

Use the SDK

Credentials

The SDK supports two types of credentials that correspond to the two connection types:

X.509 certificate

For the certificate-based mutual authentication connection type. Download the AWS IoT root CA. Use the AWS IoT console to create and download the certificate and private key. You must specify the location of these files when you initialize the client.
IAM credentials

For the Websocket with Signature Version 4 authentication type. You will need IAM credentials: an access key ID, a secret access key, and an optional session token. You must also download the AWS IoT root CA. You can specify the IAM credentails by:
- Passing method parameters
  
  The SDK will first call the following method to check if there is any input for a custom IAM credentials configuration:
```
# AWS IoT MQTT Client
AWSIoTPythonSDK.MQTTLib.AWSIoTMQTTClient.configureIAMCredentials(obtainedAccessKeyID, obtainedSecretAccessKey, obtainedSessionToken)
# AWS IoT MQTT Shadow Client
AWSIoTPythonSDK.MQTTLib.AWSIoTMQTTShadowClient.configureIAMCredentials(obtainedAccessKeyID, obtainedSecretAccessKey, obtainedSessionToken)
```
  Note: We do not recommend hard-coding credentials in a custom script. You can use Amazon Cognito Identity or another credential provider.
- Exporting environment variables
  
  If there is no custom configuration through method calls, the SDK will then check these environment variables for credentials:
  
  AWS_ACCESS_KEY_ID
  
  The access key for your AWS account.
  
  AWS_SECRET_ACCESS_KEY
  
  The secret key for your AWS account.
  
  AWS_SESSION_TOKEN
  
  The session key for your AWS account. This is required only when you are using temporary credentials. For more information, see here.
  
  You can set your IAM credentials as environment variables by using the preconfigured names. For Unix systems, you can do the following:
```
export AWS_ACCESS_KEY_ID=<your aws access key id string>
export AWS_SECRET_ACCESS_KEY=<your aws secret access key string>
export AWS_SESSION_TOKEN=<your aws session token string>
```
  For Windows, open Control Panel and choose System. In Advanced system settings choose Environment Variables and then configure the required environment variables.
- Configuring shared credentials file
  
  If there are no such environment variables specified, the SDK will check the default section for a shared credentials file (in Unix, ~/.aws/credentials and in Windows, %UserProfile%\.aws\credentials) as follows:
```
[default]
aws_access_key_id=foo
aws_secret_access_key=bar
aws_session_token=baz
```
  You can use the AWS CLI to configure the shared credentials file <http://aws.amazon.com/cli/>`__:
```
aws configure
```

AWSIoTMQTTClient

This is the client class used for plain MQTT communication with AWS IoT. You can initialize and configure the client like this:

# Import SDK packages
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient

# For certificate based connection
myMQTTClient = AWSIoTMQTTClient("myClientID")
# For Websocket connection
# myMQTTClient = AWSIoTMQTTClient("myClientID", useWebsocket=True)
# Configurations
# For TLS mutual authentication
myMQTTClient.configureEndpoint("YOUR.ENDPOINT", 8883)
# For Websocket
# myMQTTClient.configureEndpoint("YOUR.ENDPOINT", 443)
myMQTTClient.configureCredentials("YOUR/ROOT/CA/PATH", "PRIVATE/KEY/PATH", "CERTIFICATE/PATH")
# For Websocket, we only need to configure the root CA
# myMQTTClient.configureCredentials("YOUR/ROOT/CA/PATH")
myMQTTClient.configureOfflinePublishQueueing(-1)  # Infinite offline Publish queueing
myMQTTClient.configureDrainingFrequency(2)  # Draining: 2 Hz
myMQTTClient.configureConnectDisconnectTimeout(10)  # 10 sec
myMQTTClient.configureMQTTOperationTimeout(5)  # 5 sec
...

For basic MQTT operations, your script will look like this:

...
myMQTTClient.connect()
myMQTTClient.publish("myTopic", "myPayload", 0)
myMQTTClient.subscribe("myTopic", 1, customCallback)
myMQTTClient.unsubscribe("myTopic")
myMQTTClient.disconnect()
...

AWSIoTShadowClient

This is the client class used for device shadow operations with AWS IoT. You can initialize and configure the client like this:

from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTShadowClient

# For certificate based connection
myShadowClient = AWSIoTMQTTShadowClient("myClientID")
# For Websocket connection
# myMQTTClient = AWSIoTMQTTClient("myClientID", useWebsocket=True)
# Configurations
# For TLS mutual authentication
myShadowClient.configureEndpoint("YOUR.ENDPOINT", 8883)
# For Websocket
# myShadowClient.configureEndpoint("YOUR.ENDPOINT", 443)
myShadowClient.configureCredentials("YOUR/ROOT/CA/PATH", "PRIVATE/KEY/PATH", "CERTIFICATE/PATH")
# For Websocket, we only need to configure the root CA
# myShadowClient.configureCredentials("YOUR/ROOT/CA/PATH")
myShadowClient.configureConnectDisconnectTimeout(10)  # 10 sec
myShadowClient.configureMQTTOperationTimeout(5)  # 5 sec
...

For shadow operations, your script will look like this:

...
myShadowClient.connect()
# Create a device shadow instance using persistent subscription
myDeviceShadow = myShadowClient.createShadowHandlerWithName("Bot", True)
# Shadow operations
myDeviceShadow.shadowGet(customCallback, 5)
myDeviceShadow.shadowUpdate(myJSONPayload, customCallback, 5)
myDeviceShadow.shadowDelete(customCallback, 5)
myDeviceShadow.shadowRegisterDeltaCallback(customCallback)
myDeviceShadow.shadowUnregisterDeltaCallback()
...

You can also retrieve the MQTTClient(MQTT connection) to perform plain MQTT operations along with shadow operations:

myMQTTClient = myShadowClient.getMQTTConnection()
myMQTTClient.publish("plainMQTTTopic", "Payload", 1)

DiscoveryInfoProvider

This is the client class for device discovery process with AWS IoT Greengrass. You can initialize and configure the client like this:

from AWSIoTPythonSDK.core.greengrass.discovery.providers import DiscoveryInfoProvider

discoveryInfoProvider = DiscoveryInfoProvider()
discoveryInfoProvider.configureEndpoint("YOUR.IOT.ENDPOINT")
discoveryInfoProvider.configureCredentials("YOUR/ROOT/CA/PATH", "CERTIFICATE/PATH", "PRIVATE/KEY/PATH")
discoveryInfoProvider.configureTimeout(10)  # 10 sec

To perform the discovery process for a Greengrass Aware Device (GGAD) that belongs to a deployed group, your script should look like this:

discoveryInfo = discoveryInfoProvider.discover("myGGADThingName")
# I know nothing about the group/core I want to connect to. I want to iterate through all cores and find out.
coreList = discoveryInfo.getAllCores()
groupIdCAList = discoveryInfo.getAllCas()  # list([(groupId, ca), ...])
# I know nothing about the group/core I want to connect to. I want to iterate through all groups and find out.
groupList = discoveryInfo.getAllGroups()
# I know exactly which group, which core and which connectivity info I need to connect.
connectivityInfo = discoveryInfo.toObjectAtGroupLevel()["YOUR_GROUP_ID"]
                                .getCoreConnectivityInfo("YOUR_CORE_THING_ARN")
                                .getConnectivityInfo("YOUR_CONNECTIVITY_ID")
# Connecting logic follows...
...

For more information about discovery information access at group/core/connectivity info set level, please refer to the API documentation for AWSIoTPythonSDK.core.greengrass.discovery.models, Greengrass Discovery documentation or Greengrass overall documentation.

Key Features

Progressive Reconnect Back Off

When a non-client-side disconnect occurs, the SDK will reconnect automatically. The following APIs are provided for configuration:

# AWS IoT MQTT Client
AWSIoTPythonSDK.MQTTLib.AWSIoTMQTTClient.configureAutoReconnectBackoffTime(baseReconnectQuietTimeSecond, maxReconnectQuietTimeSecond, stableConnectionTimeSecond)
# AWS IoT MQTT Shadow Client
AWSIoTPythonSDK.MQTTLib.AWSIoTMQTTShadowClient.configureAutoReconnectBackoffTime(baseReconnectQuietTimeSecond, maxReconnectQuietTimeSecond, stableConnectionTimeSecond)

The auto-reconnect occurs with a progressive backoff, which follows this mechanism for reconnect backoff time calculation:

t^current = min(2ⁿ t^base, t^max)

where t^current is the current reconnect backoff time, t^base is the base reconnect backoff time, t^max is the maximum reconnect backoff time.

The reconnect backoff time will be doubled on disconnect and reconnect attempt until it reaches the preconfigured maximum reconnect backoff time. After the connection is stable for over the stableConnectionTime, the reconnect backoff time will be reset to the baseReconnectQuietTime.

If no configureAutoReconnectBackoffTime is called, the following default configuration for backoff timing will be performed on initialization:

baseReconnectQuietTimeSecond = 1
maxReconnectQuietTimeSecond = 32
stableConnectionTimeSecond = 20

Offline Requests Queueing with Draining

If the client is temporarily offline and disconnected due to network failure, publish/subscribe/unsubscribe requests will be added to an internal queue until the number of queued-up requests reaches the size limit of the queue. This functionality is for plain MQTT operations. Shadow client contains time-sensitive data and is therefore not supported.

The following API is provided for configuration:

AWSIoTPythonSDK.MQTTLib.AWSIoTMQTTClient.configureOfflinePublishQueueing(queueSize, dropBehavior)

After the queue is full, offline publish/subscribe/unsubscribe requests will be discarded or replaced according to the configuration of the drop behavior:

# Drop the oldest request in the queue
AWSIoTPythonSDK.MQTTLib.DROP_OLDEST = 0
# Drop the newest request in the queue
AWSIoTPythonSDK.MQTTLib.DROP_NEWEST = 1

Let's say we configure the size of offlinePublishQueue to 5 and we have 7 incoming offline publish requests.

In a DROP_OLDEST configuration:

myClient.configureOfflinePublishQueueing(5, AWSIoTPythonSDK.MQTTLib.DROP_OLDEST);

The internal queue should be like this when the queue is just full:

HEAD ['pub_req1', 'pub_req2', 'pub_req3', 'pub_req4', 'pub_req5']

When the 6th and the 7th publish requests are made offline, the internal queue will be like this:

HEAD ['pub_req3', 'pub_req4', 'pub_req5', 'pub_req6', 'pub_req7']

Because the queue is already full, the oldest requests pub_req1 and pub_req2 are discarded.

In a DROP_NEWEST configuration:

myClient.configureOfflinePublishQueueing(5, AWSIoTPythonSDK.MQTTLib.DROP_NEWEST);

The internal queue should be like this when the queue is just full:

HEAD ['pub_req1', 'pub_req2', 'pub_req3', 'pub_req4', 'pub_req5']

When the 6th and the 7th publish requests are made offline, the internal queue will be like this:

HEAD ['pub_req1', 'pub_req2', 'pub_req3', 'pub_req4', 'pub_req5']

Because the queue is already full, the newest requests pub_req6 and pub_req7 are discarded.

When the client is back online, connected, and resubscribed to all topics it has previously subscribed to, the draining starts. All requests in the offline request queue will be resent at the configured draining rate:

AWSIoTPythonSDK.MQTTLib.AWSIoTMQTTClient.configureDrainingFrequency(frequencyInHz)

If no configOfflinePublishQueue or configureDrainingFrequency is called, the following default configuration for offline request queueing and draining will be performed on the initialization:

offlinePublishQueueSize = 20
dropBehavior = DROP_NEWEST
drainingFrequency = 2Hz

Before the draining process is complete, any new publish/subscribe/unsubscribe request within this time period will be added to the queue. Therefore, the draining rate should be higher than the normal request rate to avoid an endless draining process after reconnect.

The disconnect event is detected based on PINGRESP MQTT packet loss. Offline request queueing will not be triggered until the disconnect event is detected. Configuring a shorter keep-alive interval allows the client to detect disconnects more quickly. Any QoS0 publish, subscribe and unsubscribe requests issued after the network failure and before the detection of the PINGRESP loss will be lost.

Persistent/Non-Persistent Subscription

Device shadow operations are built on top of the publish/subscribe model for the MQTT protocol, which provides an asynchronous request/response workflow. Shadow operations (Get, Update, Delete) are sent as requests to AWS IoT. The registered callback will be executed after a response is returned. In order to receive responses, the client must subscribe to the corresponding shadow response topics. After the responses are received, the client might want to unsubscribe from these response topics to avoid getting unrelated responses for charges for other requests not issued by this client.

The SDK provides a persistent/non-persistent subscription selection on the initialization of a device shadow. Developers can choose the type of subscription workflow they want to follow.

For a non-persistent subscription, you will need to create a device shadow like this:

nonPersistentSubShadow = myShadowClient.createShadowHandlerWithName("NonPersistentSubShadow", False)

In this case, the request to subscribe to accepted/rejected topics will be sent on each shadow operation. After a response is returned, accepted/rejected topics will be unsubscribed to avoid getting unrelated responses.

For a persistent subscription, you will need to create a device shadow like this:

persistentSubShadow = myShadowClient.createShadowHandlerWithName("PersistentSubShadow", True)

In this case, the request to subscribe to the corresponding accepted/rejected topics will be sent on the first shadow operation. For example, on the first call of shadowGet API, the following topics will be subscribed to on the first Get request:

$aws/things/PersistentSubShadow/shadow/get/accepted
$aws/things/PersistentSubShadow/shadow/get/rejected

Because it is a persistent subscription, no unsubscribe requests will be sent when a response is returned. The SDK client is always listening on accepted/rejected topics.

In all SDK examples, PersistentSubscription is used in consideration of its better performance.

Examples

BasicPubSub

This example demonstrates a simple MQTT publish/subscribe using AWS IoT. It first subscribes to a topic and registers a callback to print new messages and then publishes to the same topic in a loop. New messages are printed upon receipt, indicating the callback function has been called.

Instructions

Run the example like this:

# Certificate based mutual authentication
python basicPubSub.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath>
# MQTT over WebSocket
python basicPubSub.py -e <endpoint> -r <rootCAFilePath> -w
# Customize client id and topic
python basicPubSub.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath> -id <clientId> -t <topic>

Source

The example is available in samples/basicPubSub/.

BasicPubSub with Amazon Cognito Session Token

This example demonstrates a simple MQTT publish/subscribe using an Amazon Cognito Identity session token. It uses the AWS IoT Device SDK for Python and the AWS SDK for Python (boto3). It first makes a request to Amazon Cognito to retrieve the access ID, the access key, and the session token for temporary authentication. It then uses these credentials to connect to AWS IoT and communicate data/messages using MQTT over Websocket, just like the BasicPubSub example.

Instructions

To run the example, you will need your Amazon Cognito identity pool ID and allow unauthenticated identities to connect. Make sure that the policy attached to the unauthenticated role has permissions to access the required AWS IoT APIs. For more information about Amazon Cognito, see here.

Run the example like this:

python basicPubSub_CognitoSTS.py -e <endpoint> -r <rootCAFilePath> -C <CognitoIdentityPoolID>
# Customize client id and topic
python basicPubsub_CognitoSTS.py -e <endpoint> -r <rootCAFilePath> -C <CognitoIdentityPoolID> -id <clientId> -t <topic>

Source

The example is available in samples/basicPubSub/.

BasicPubSub Asynchronous version

This example demonstrates a simple MQTT publish/subscribe with asynchronous APIs using AWS IoT. It first registers general notification callbacks for CONNACK reception, disconnect reception and message arrival. It then registers ACK callbacks for subscribe and publish requests to print out received ack packet ids. It subscribes to a topic with no specific callback and then publishes to the same topic in a loop. New messages are printed upon reception by the general message arrival callback, indicating the callback function has been called. New ack packet ids are printed upon reception of PUBACK and SUBACK through ACK callbacks registered with asynchronous API calls, indicating that the the client received ACKs for the corresponding asynchronous API calls.

Instructions

Run the example like this:

# Certificate based mutual authentication
python basicPubSubAsync.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath>
# MQTT over WebSocket
python basicPubSubAsync.py -e <endpoint> -r <rootCAFilePath> -w
# Customize client id and topic
python basicPubSubAsync.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath> -id <clientId> -t <topic>

Source

The example is available in samples/basicPubSub/.

BasicShadow

This example demonstrates the use of basic shadow operations (update/delta). It has two scripts, basicShadowUpdater.py and basicShadowDeltaListener.py. The example shows how an shadow update request triggers delta events.

basicShadowUpdater.py performs a shadow update in a loop to continuously modify the desired state of the shadow by changing the value of the integer attribute.

basicShadowDeltaListener.py subscribes to the delta topic of the same shadow and receives delta messages when there is a difference between the desired and reported states.

Because only the desired state is being updated by basicShadowUpdater, a series of delta messages that correspond to the shadow update requests should be received in basicShadowDeltaListener.

Instructions

Run the example like this:

First, start the basicShadowDeltaListener:

# Certificate-based mutual authentication
python basicShadowDeltaListener.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath>
# MQTT over WebSocket
python basicShadowDeltaListener.py -e <endpoint> -r <rootCAFilePath> -w

Then, start the basicShadowUpdater:

# Certificate-based mutual authentication
python basicShadowUpdater.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath>
# MQTT over WebSocket
python basicShadowUpdater.py -e <endpoint> -r <rootCAFilePath> -w

After the basicShadowUpdater starts sending shadow update requests, you should be able to see corresponding delta messages in the basicShadowDeltaListener output.

Source

The example is available in samples/basicShadow/.

ThingShadowEcho

This example demonstrates how a device communicates with AWS IoT, syncing data into the device shadow in the cloud and receiving commands from another app. Whenever there is a new command from the app side to change the desired state of the device, the device receives this request and applies the change by publishing it as the reported state. By registering a delta callback function, users will be able to see this incoming message and notice the syncing of the state.

Instructions

Run the example like this:

# Certificate based mutual authentication
python ThingShadowEcho.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath>
# MQTT over WebSocket
python ThingShadowEcho.py -e <endpoint> -r <rootCAFilePath> -w
# Customize client Id and thing name
python ThingShadowEcho.py -e <endpoint> -r <rootCAFilePath> -c <certFilePath> -k <privateKeyFilePath> -id <clientId> -n <thingName>

Now use the AWS IoT console or other MQTT client to update the shadow desired state only. You should be able to see the reported state is updated to match the changes you just made in desired state.

Source

The example is available in samples/ThingShadowEcho/.

BasicDiscovery

This example demonstrates how to perform a discovery process from a Greengrass Aware Device (GGAD) to obtain the required connectivity/identity information to connect to the Greengrass Core (GGC) deployed within the same group. It uses the discovery information provider to invoke discover call for a certain GGAD with its thing name. After it gets back a success response, it picks up the first GGC and the first set of identity information (CA) for the first group, persists it locally and iterates through all connectivity info sets for this GGC to establish a MQTT connection to the designated GGC. It then publishes messages to the topic, which, on the GGC side, is configured to route the messages back to the same GGAD. Therefore, it receives the published messages and invokes the corresponding message callbacks.

Note that in order to get the sample up and running correctly, you need:

Have a successfully deployed Greengrass group.
Use the certificate and private key that have been deployed with the group for the GGAD to perform discovery process.
The subscription records for that deployed group should contain a route that routes messages from the targeted GGAD to itself via a dedicated MQTT topic.
The deployed GGAD thing name, the deployed GGAD certificate/private key and the dedicated MQTT topic should be used as the inputs for this sample.

Run the sample like this:

python basicDiscovery.py -e <endpoint> -r <IoTRootCAFilePath> -c <certFilePath> -k <privateKeyFilePath> -n <GGADThingName> -t <RoutingTopic>

If the group, GGC, GGAD and group subscription/routes are set up correctly, you should be able to see the sample running on your GGAD, receiving messages that get published to GGC by itself.

API Documentation

You can find the API documentation for the SDK here.

License

This SDK is distributed under the Apache License, Version 2.0, see LICENSE.txt and NOTICE.txt for more information.

Support

If you have technical questions about the AWS IoT Device SDK, use the AWS IoT Forum. For any other questions about AWS IoT, contact AWS Support.

License

frankwang5168/aws-iot-device-sdk-python

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages