diff --git a/README.rst b/README.rst index 5cf93962..372f5a91 100644 --- a/README.rst +++ b/README.rst @@ -1,49 +1,43 @@ -.. Copyright 2023 ClearBlade Inc. - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - Copyright 2022 Google LLC - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - -Python Client for ClearBlade Internet of Things (IoT) Core API +Python Client for ClearBlade IoT Core API ================================================================ -Quick Start +Quick start ----------- +**Notes** -In order to use this library, you first need to go through the following steps: +1. This SDK is for use with ClearBlade IoT Core Standard and ClearBlade IoT Core Enterprise. + +2. ClearBlade IoT **Core** Enterprise is DIFFERENT from ClearBlade IoT Enterprise!!! + +3. If you are using ClearBlade but not using ClearBlade IoT Core (Standard or Enterprise), then you need this different SDK: ``_. + +4. To understand how IoT Core Enterprise differs from Standard, look here: ``_. + + + +To use this SDK, you first need to go through the following steps: 1. Install pip package - ```pip install clearblade-cloud-iot``` +2. Set an environment variable **CLEARBLADE_CONFIGURATION**, pointing to your ClearBlade service account JSON file (see below). + +3. Optionally set an environment variable **BINARYDATA_AND_TIME_GOOGLE_FORMAT** to True. Look at **Note about types of times and binaryData** below for details. + + +To create a service account and download a credentials JSON file look at the appropriate link below: + +`ClearBlade IoT Core Standard `_ + +`ClearBlade IoT Core Enterprise `_ -2. Set an environment variable **CLEARBLADE_CONFIGURATION** which should point to your clearblade service account json file. -3. Optionally set an environment variable **BINARYDATA_AND_TIME_GOOGLE_FORMAT** to True. Look at **Note about types of times and binaryData** below for details. Installation ~~~~~~~~~~~~ -Install this library in a `virtualenv`_ using pip. `virtualenv`_ is a tool to -create isolated Python environments. The basic problem it addresses is one of -dependencies and versions, and indirectly permissions. +Install this library in a `virtualenv`_ using pip. `virtualenv`_ is a tool to create isolated Python environments. It addresses dependencies and versions and, indirectly, permissions. -With `virtualenv`_, it's possible to install this library without needing system -install permissions, and without clashing with the installed system -dependencies. +With `virtualenv`_, it's possible to install this library without system install permissions and clashing with the installed system dependencies. .. _`virtualenv`: https://virtualenv.pypa.io/en/latest/ @@ -51,10 +45,10 @@ dependencies. Code samples and snippets ~~~~~~~~~~~~~~~~~~~~~~~~~ -Code samples and snippets live in the `samples/clearblade` folder. +Code samples and snippets live in the samples/clearblade folder. -Supported Python Versions +Supported Python versions ^^^^^^^^^^^^^^^^^^^^^^^^^ Our client libraries are compatible with all current `active`_ and `maintenance`_ versions of Python. @@ -64,12 +58,11 @@ Python >= 3.7 .. _active: https://devguide.python.org/devcycle/#in-development-main-branch .. _maintenance: https://devguide.python.org/devcycle/#maintenance-branches -Unsupported Python Versions +Unsupported Python versions ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Python <= 3.6 -If you are using an `end-of-life`_ -version of Python, we recommend that you update as soon as possible to an actively supported version. +If you are using an `end-of-life`_ version of Python, we recommend you update it to an actively supported version as soon as possible. .. _end-of-life: https://devguide.python.org/devcycle/#end-of-life-branches @@ -92,19 +85,19 @@ Windows virtualenv \Scripts\activate -Next Steps +Next steps ~~~~~~~~~~ -- clone the github repository. +- Clone the GitHub repository. -- and execute the setup.py file like , python setup.py install. +- Execute the setup.py file like Python setup.py install. -- mostly if you change you imports from from google.cloud to clearblade.cloud everything else should work. +- Everything else should work if you change your imports from google.cloud to clearblade.cloud. Note about types of times and binaryData ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- By default the following parameters are returned as the shown types: +- By default, the following parameters are returned as the shown types: 1. All time parameters (e.g. **cloudUpdateTime**, **deviceAckTime**, **updateTime**): **RFC3339** strings (e.g. "2023-01-12T23:38:07.732Z") 2. **CONFIG binaryData**: **base64-encoded string** @@ -116,20 +109,30 @@ Note about types of times and binaryData 1. All times: **DatetimeWithNanoseconds** (defined in the **proto.datetime_helpers** module) 2. All **binaryData** (CONFIG, STATE etc.): **BYTE ARRAYS** -- If this environment variable is not set, or is set to any unexpeced values, then the default types listed previously are used. +- If this environment variable is not set, or is set to any unexpected values, then the default types listed previously are used. + +Note about performance: +~~~~~~~~~~~~~~~~~~~~~~~ + +- By default, calls to some SDK functions cause a REST request to be sent to acquire the registry API keys found on the IoTCore UI Registry Details page. Those keys are cached for subsequent operations to improve performance. However, these caches do not persist if the application is stopped and restarted, as would be the case with typical serverless functions (e.g., Google Cloud Functions, AWS Lambda, etc.). To improve those functions' performance, the REST call can be prevented by passing the API keys as environment variables: + +1. **REGISTRY_URL**: **string** +2. **REGISTRY_SYSKEY**: **string** +3. **REGISTRY_TOKEN**: **string** + +Note about running from the source instead of the PyPi (pip) module: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Note about running from source instead of PyPi (pip) module: -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- To temporarily use the source code in this repo. instead of the installed PyPi (pip) module do the following: +- To temporarily use the source code in this repo instead of the installed PyPi (pip) module, do the following: 1. Clone this repo. -2. Checkout the desired branch using **git checkout **. +2. Check out the desired branch using **git checkout **. 3. In your code find where **clearblade** or **clearblade.cloud** is being imported. -4. Precede that line with **import sys** and **sys.path.insert(0, )**. The path must end with "python-iot". So for example: +4. Precede that line with **import sys** and **sys.path.insert(0, )**. The path must end with python-iot. For example: .. code-block:: console import sys sys.path.insert(0, "path/to/python-iot") - from clearblade.cloud import iot_v1 \ No newline at end of file + from clearblade.cloud import iot_v1 diff --git a/UPGRADING.md b/UPGRADING.md index 1b711d5a..74ba8d77 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -42,7 +42,7 @@ limitations under the License. --> # 2.0.0 Migration Guide -The 2.0 release of the `clearblade-cloud-iot` client is a significant upgrade based on addition of two new classes in **iot_v1**: +The 2.0 release of the `clearblade-cloud-iot` client is a significant upgrade based on the addition of two new classes in **iot_v1**: - **DeviceCredential** - **PublicKeyCredential** @@ -52,7 +52,7 @@ The release also includes enhancements to these classes already present in **iot - **DeviceConfig** - **DeviceState** -The version was made with the intent of minimizing required code changes. **However these changes should be considrered Breaking changes**. +The version was made with the intent of minimizing the required code changes. **However, these changes should be considered breaking changes**. # @@ -64,13 +64,13 @@ The version was made with the intent of minimizing required code changes. **Howe **After**: device.credentials is of type **[DeviceCredential]** (i.e. list of objects of class DeviceCredential). - The **DeviceCredential** class has these features for usability: + The **DeviceCredential** class has these usability features: - - A **get** method that mimics the **get** method of a dict. + - A **get** method that mimics the dict **get** method. - Allows accessing attributes using dot notation OR square-brackets. - - Supports camel-case as well as snake-case for accessing attributes: + - Supports camel-case and snake-case for accessing attributes: - e.g. All these are valid for retrieving the public key: + e.g., All these are valid for retrieving the public key: - **public_key = device.credentials[0]['publicKey']** - **public_key = device.credentials[0]['public_key']** @@ -81,7 +81,7 @@ The version was made with the intent of minimizing required code changes. **Howe # -2. This refers to pub_key mentioned in the previous section. +2. This refers to pub_key mentioned in the previous section: **Before**: public_key was of type **dict**. @@ -89,12 +89,12 @@ The version was made with the intent of minimizing required code changes. **Howe **After**: public_key is an object of class **PublicKeyCredential**. - The **PublicKeyCredential** class has these features for usability: + The **PublicKeyCredential** class has these usability features: - - A **get** method that mimics the **get** method of a dict. + - A **get** method that mimics the dict **get** method. - Allows accessing attributes using dot notation OR square-brackets. - e.g. All these are valid for retrieving the public key format: + e.g., All these are valid for retrieving the public key format: - **format = public_key['format']** - **format = public_key.get('format')** @@ -110,13 +110,13 @@ The version was made with the intent of minimizing required code changes. **Howe **After**: dev_config is an object of class **DeviceConfig**. - The **DeviceConfig** class has these features for usability: + The **DeviceConfig** class has these usability features: - - A **get** method that mimics the **get** method of a dict. + - A **get** method that mimics the dict **get** method. - Allows accessing attributes using dot notation OR square-brackets. - - Supports camel-case as well as snake-case for accessing attributes: + - Supports camel-case and snake-case for accessing attributes: - e.g. All these are valid for retrieving the cloud_update_time: + e.g., All these are valid for retrieving the cloud_update_time: - **cloud_update_time = device.credentials[0]['cloudUpdateTime']** - **cloud_update_time = device.credentials[0]['cloud_update_time']** @@ -135,13 +135,13 @@ The version was made with the intent of minimizing required code changes. **Howe **After**: dev_state is an object of class **DeviceState**. - The **DeviceState** class has these features for usability: + The **DeviceState** class has these usability features: - - A **get** method that mimics the **get** method of a dict. + - A **get** method that mimics the dict **get** method. - Allows accessing attributes using dot notation OR square-brackets. - - Supports camel-case as well as snake-case for accessing attributes: + - Supports camel-case and snake-case for accessing attributes: - e.g. All these are valid for retrieving the binary_data: + e.g., All these are valid for retrieving the binary_data: - **binary_data = device.credentials[0]['binaryData']** - **binary_data = device.credentials[0]['binary_data']** diff --git a/clearblade/cloud/iot_v1/config_manager.py b/clearblade/cloud/iot_v1/config_manager.py index fb8cd559..4ac94e1b 100644 --- a/clearblade/cloud/iot_v1/config_manager.py +++ b/clearblade/cloud/iot_v1/config_manager.py @@ -82,6 +82,11 @@ def _set_admin_clearblade_config(self): self._admin_config = ClearBladeConfig(system_key=system_key, auth_token=auth_token, api_url=api_url, project=project) + def _create_regional_config_from_env(self, systemKey:str = None, serviceAccountToken:str = None, url:str = None, region:str = None)-> ClearBladeConfig : + return ClearBladeConfig(system_key=systemKey, auth_token=serviceAccountToken, api_url=url, + region=region, project=self._admin_config.project) + + def _create_regional_config(self, regional_json: json = None)-> ClearBladeConfig : system_key = regional_json['systemKey'] auth_token = regional_json['serviceAccountToken'] @@ -102,20 +107,25 @@ def _set_regional_config(self, region:str = None, registry:str = None): if not region or not registry: raise Exception("Either location or registry name is not provided") - - sync_client = SyncClient(clearblade_config=self._admin_config) - request_body = {'region':region,'registry':registry, 'project':self._admin_config.project} - response = sync_client.post(api_name="getRegistryCredentials", is_webhook_folder=False, - request_body=request_body) - - if response.status_code != 200: - raise Exception( - f"\n\nRegistry Information not found! Please check if the given registry exists\nProject: {self._admin_config.project}\nRegistry: {registry}\nRegion: {region}" - ) - - response_json = response.json() - response_json['region'] = region - self._regional_config = self._create_regional_config(regional_json=response_json) + + system_key = os.environ.get("REGISTRY_SYSKEY") + registry_token = os.environ.get("REGISTRY_TOKEN") + url = os.environ.get("REGISTRY_URL") + if (system_key and registry_token and url) : + self._regional_config = self._create_regional_config_from_env(systemKey=system_key , serviceAccountToken = registry_token, url = url, region = region) + else : + request_body = {'region':region,'registry':registry, 'project':self._admin_config.project} + sync_client = SyncClient(clearblade_config=self._admin_config) + response = sync_client.post(api_name="getRegistryCredentials", is_webhook_folder=False, + request_body=request_body) + if response.status_code != 200: + raise Exception( + f"\n\nRegistry Information not found! Please check if the given registry exists\nProject: {self._admin_config.project}\nRegistry: {registry}\nRegion: {region}" + ) + response_json = response.json() + response_json['region'] = region + self._regional_config = self._create_regional_config(regional_json=response_json) + async def _set_regional_config_async(self, region:str = None, registry:str = None): @@ -126,21 +136,25 @@ async def _set_regional_config_async(self, region:str = None, registry:str = Non if not registry: registry = self.registry_name - - async_client = AsyncClient(clearblade_config=self._admin_config) - request_body = {'region':region,'registry':registry, 'project':self._admin_config.project} - response = await async_client.post(api_name="getRegistryCredentials", - is_webhook_folder=False, - request_body=request_body) - - if response.status_code != 200: - raise Exception( - f"\n\nRegistry Information not found! Please check if the given registry exists\nProject: {self._admin_config.project}\nRegistry: {registry}\nRegion: {region}" - ) - - response_json = response.json() - response_json['region'] = region - self._regional_config = self._create_regional_config(regional_json=response_json) + system_key = os.environ.get("REGISTRY_SYSKEY") + registry_token = os.environ.get("REGISTRY_TOKEN") + url = os.environ.get("REGISTRY_URL") + if (system_key and registry_token) : + self._regional_config = self._create_regional_config_from_env(systemKey=system_key , serviceAccountToken = registry_token, url = url, region = region) + else : + async_client = AsyncClient(clearblade_config=self._admin_config) + request_body = {'region':region,'registry':registry, 'project':self._admin_config.project} + response = await async_client.post(api_name="getRegistryCredentials", + is_webhook_folder=False, + request_body=request_body) + + if response.status_code != 200: + raise Exception( + f"\n\nRegistry Information not found! Please check if the given registry exists\nProject: {self._admin_config.project}\nRegistry: {registry}\nRegion: {region}" + ) + response_json = response.json() + response_json['region'] = region + self._regional_config = self._create_regional_config(regional_json=response_json) @property diff --git a/clearblade/cloud/iot_v1/device_types.py b/clearblade/cloud/iot_v1/device_types.py index dd21121a..0b7d6cdd 100644 --- a/clearblade/cloud/iot_v1/device_types.py +++ b/clearblade/cloud/iot_v1/device_types.py @@ -43,7 +43,7 @@ """ from typing import List -from .resources import GatewayType, LogLevel, PublicKeyFormat, PublicKeyCredential, DeviceCredential +from .resources import GatewayType, LogLevel, PublicKeyFormat, PublicKeyCredential, DeviceCredential, FieldMask from .utils import get_value import os from proto.datetime_helpers import DatetimeWithNanoseconds @@ -73,7 +73,9 @@ def __init__(self, id: str, num_id: str = None, last_error_time: str = None, last_error_status_code: dict = None, config: dict = {"cloudUpdateTime":None, "version":""} , state: dict = {"updateTime":None, "binaryData":None}, - log_level: str = LogLevel.NONE, meta_data: dict = {}, gateway_config : dict = {"gatewayType": GatewayType.NON_GATEWAY}) -> None: + log_level: str = LogLevel.LOG_LEVEL_UNSPECIFIED, + meta_data: dict = {}, + gateway_config: dict = {"gatewayType": GatewayType.NON_GATEWAY}) -> None: self._id = id self._name = '' @@ -589,7 +591,7 @@ def _prepare_params_for_list(self): if self.device_ids: params['deviceIds'] = self.device_ids if self.field_mask: - params['fieldMask'] = self.field_mask + params['fieldMask'] = FieldMask.convert_fieldmask_for_list(self.field_mask) if self.gateway_list_options : if 'associationsDeviceId' in self.gateway_list_options: params['gatewayListOptions.associationsDeviceId'] = self.gateway_list_options['associationsDeviceId'] @@ -634,4 +636,4 @@ def from_json(devices_list_json): if 'nextPageToken' in devices_list_json: next_page_token = devices_list_json['nextPageToken'] - return ListDevicesResponse(devices=devices, next_page_token=next_page_token) + return ListDevicesResponse(devices=devices, next_page_token=next_page_token) \ No newline at end of file diff --git a/clearblade/cloud/iot_v1/devices.py b/clearblade/cloud/iot_v1/devices.py index 97633860..244492e9 100644 --- a/clearblade/cloud/iot_v1/devices.py +++ b/clearblade/cloud/iot_v1/devices.py @@ -490,4 +490,4 @@ async def config_versions_list_async(self, if response.status_code == 200: return ListDeviceConfigVersionsResponse.from_json(response.json()) - return response + return response \ No newline at end of file diff --git a/clearblade/cloud/iot_v1/resources.py b/clearblade/cloud/iot_v1/resources.py index b933a701..0d37e590 100644 --- a/clearblade/cloud/iot_v1/resources.py +++ b/clearblade/cloud/iot_v1/resources.py @@ -168,4 +168,22 @@ def convert_credentials_for_create_update(cls, credentials): if updateDeviceCredential: credentials[index] = credential - return credentials \ No newline at end of file + return credentials + +class FieldMask(): + def __init__(self, paths: [str]): + self.paths = paths + + def __getitem__(self, arg): + return getattr(self, arg) + + def get(self, arg): + return getattr(self, arg) + + @classmethod + def convert_fieldmask_for_list(cls, field_mask): + if (isinstance(field_mask, FieldMask)): + field_mask = field_mask.__dict__ + if 'paths' in field_mask: + field_mask = field_mask['paths'] + return field_mask diff --git a/google/cloud/iot_v1/types/device_manager.py b/google/cloud/iot_v1/types/device_manager.py index 93d6622d..8b8b19f5 100644 --- a/google/cloud/iot_v1/types/device_manager.py +++ b/google/cloud/iot_v1/types/device_manager.py @@ -318,11 +318,11 @@ class ListDevicesRequest(proto.Message): ``projects/my-project/locations/us-central1/registries/my-registry``. device_num_ids (Sequence[int]): A list of device numeric IDs. If empty, this - field is ignored. Maximum IDs: 10,000. + field is ignored. device_ids (Sequence[str]): A list of device string IDs. For example, ``['device0', 'device12']``. If empty, this field is - ignored. Maximum IDs: 10,000 + ignored. field_mask (google.protobuf.field_mask_pb2.FieldMask): The fields of the ``Device`` resource to be returned in the response. The fields ``id`` and ``num_id`` are always diff --git a/setup.py b/setup.py index 06f2af21..56ba1951 100644 --- a/setup.py +++ b/setup.py @@ -32,7 +32,7 @@ name = "clearblade-cloud-iot" description = "Cloud IoT API client library" -version = "2.0.2" +version = "2.0.7" release_status = "Development Status :: 5 - Production/Stable" dependencies = ["httpx", "proto-plus"] @@ -58,7 +58,7 @@ description=description, long_description=readme, author="Clearblade", - author_email="googleapis-packages@oogle.com", + author_email="info@clearblade.com", license="Apache 2.0", url="https://github.com/clearblade/python-iot", classifiers=[