This repository contains some simple Python scripts to showcase how to use our GraphQL API.
Read our Zeenea GraphQL Catalog API documentation for details.
List of the scripts:
- export_items_in_excel.py: Export some Zeenea items into an Excel file.
- update_items_from_excel.py: Update Zeenea items from an Excel file containing a property and a description.
- send_dqm_results.py: Inject DQM results for Zeenea datasets from an external source.
- send_field_lineage.py: Inject Field to Field lineage from an external source.
- user.py: A command line tool to create, modify and delete users.
- migrate_contact.py: A command line tool to copy contact-items links of a contact to another one.
To set up the project:
- Install a Python environment if needed.
- Clone this repository.
- Create a virtual environment with venv.
- Activate the new virtual environment.
- Install the dependencies listed in requirements.txt.
- Prepare the settings. You have two options to do this:
- Edit settings.toml and .secrets.toml files.
- Use the
zeenea.setupwizard to guide you in the settings.
- Your environment is ready you can run the examples.
Scripts have been tested with Python 3.12.
> py -m venv venv
> venv/Scripts/activate
> py -m pip install -r requirements.txt
> py -m zeenea.setup
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ python -m zeenea.setup
NOTE: you may have to use python3instead of python if using MacOS
In order to make easier to prepare the settings, we provided you with a small command line tool. The wizard can be used several times without losing the existing settings.
You can call it with a simple command:
❯ py -m zeenea.setup
$ python -m zeenea.setup
$ python -m zeenea.setup
? Zeenea tenant: acme
? Zeenea API Secret: **************************************
? Which example do you want to try ? (Use arrow keys to move, <space> to select, <a> to toggle, <i> to invert)
» ● export_items_in_excel.py
● update_items_from_excel.py
● send_field_lineage.py
● send_dqm_results.py
* Options for export_items_in_excel.py
? Excel output file: output/datasets.xlsx
? Export page size:
* Options for update_items_from_excel.py
? Excel input file: input/datasets.xlsx
* Options for send_field_lineage.py
? Lineage input file: input/lineage.json
* Options for send_dqm_results.py
? DQM input file: input/dqm-results.csv
In order to run a script
- Set up the environment, if already set, don't forget to activate the virtual environment.
- Update settings files.
- Run the python interpreter or directly execute the command (linux/unix).
> venv/Scripts/activate
> py export_items_in_excel.py
$ source .venv/bin/activate
$ python export_items_in_excel.py
$ ./export_items_in_excel.py
Export some Zeenea items into an Excel file. (Sources)
In settings.toml:
- tenant: The tenant name. Example: "acme". For very specific use cases, a URL prefix can be provided.
- excel_output_file: The path to the Excel output file. The default value is "output/datasets.xlsx".
- page_size: The size of a page. Default to 20.
In .secrets.toml:
- api_secret: A valid Zeenea API Secret with the scope "Manage documentation".
- dynaconf: Configuration.
- httpx: HTTP request.
- pandas: Data frames manipulation.
- openpyxl: Excel writer.
Update Zeenea items from an Excel file containing a property and a description. (Sources)
In settings.toml:
- tenant: The tenant name. Example: "acme". For very specific use cases, a URL prefix can be provided.
- excel_input_file: The path to the Excel input file. The default value is "input/datasets.xlsx".
In .secrets.toml:
- api_secret: A valid Zeenea API Secret with the scope "Manage documentation".
- dynaconf: Configuration.
- httpx: HTTP request.
- pandas: Data frames manipulation.
- openpyxl: Excel reader.
Inject DQM results for Zeenea datasets from an external source. (Sources)
In settings.toml:
- tenant: The tenant name. Example: "acme". For very specific use cases, a URL prefix can be provided.
- dqm_input_file: The path to the CSV input file. The default value is "input/dqm-results.csv".
In .secrets.toml:
- api_secret: A valid Zeenea API Secret with the scope "Manage documentation".
- dynaconf: Configuration.
- httpx: HTTP request.
Inject Field to Field lineage from an external source. (Sources)
The source here is a sample JSON file input/lineage.json. In real implementation you will fetch the information directly from the source system using API or any intermediate file format depending on your context.
In settings.toml:
- tenant: The tenant name. Example: "acme". For very specific use cases, a URL prefix can be provided.
- lineage_input_file: The path to the JSON input file. The default value is "input/lineage.json".
In .secrets.toml:
- api_secret: A valid Zeenea API Secret with the scope "Manage documentation".
- dynaconf: Configuration.
- httpx: HTTP request.
This is a command line tool to create, modify or delete users.
In settings.toml:
- tenant: The tenant name. Example: "acme". For very specific use cases, a URL prefix can be provided.
In .secrets.toml:
- scim_api_secret: A valid Zeenea API Secret with the scope "Admin".
You can get the command line arguments documentation with the command line itself with the option --help.
For create command, options are:
-e,--email: The user email address.--given-name: The user given name.--family-name: The user family name.-g,--group: A group to add the user to. Examples:
> py user.py --help
usage: user.py [-h] {create,delete,modify} ...
CLI to manage users with scim as an integration example
options:
-h, --help show this help message and exit
user commands:
{create,delete,modify}
create Create a new user
delete Delete an existing user
modify Modify a user
> py .\user.py create --help
usage: user.py create [-h] -e EMAIL [--given-name GIVEN_NAME] [--family-name FAMILY_NAME] [-g GROUP]
options:
-h, --help show this help message and exit
-e EMAIL, --email EMAIL
Email address
--given-name GIVEN_NAME
Given name
--family-name FAMILY_NAME
Family name
-g GROUP, --group GROUP
Group to add the user to
- argparse: a command line argument parser.
- scim2_client: A Scim 2.0 client library.
- dynaconf: Configuration.
- httpx: HTTP request.
This is a command line tool to copy the contact-item links of a contact to another one. This can help the transmission of a responsibility from a contact to another one. This can happen when a person changes or a new user must be created for a person whose email changes.
In settings.toml:
- tenant: The tenant name. Example: "acme". For very specific use cases, a URL prefix can be provided.
In .secrets.toml:
- scim_api_secret: A valid Zeenea API Secret with the scope "Admin".
Command line options:
--from: Email address of the contact to copy from.--to: Email address of the contact to copy to.
You can get the command line arguments documentation with the command line itself with the option --help.
> py user.py create --email [email protected] --given-name John --family-name Doe
> py migrate_contact.py --from [email protected] --to [email protected]
> py user.py delete --email [email protected]
- argparse: a command line argument parser.
- dynaconf: Configuration.
- httpx: HTTP request.
The example files use common modules in the zeenea package for common technical code. You can look at these modules, reuse them. However, there are provided as examples and Zeenea provides no guaranty or support about them.
This module provides a small client to make easier to use the Zeenea GraphQL API. It's mostly a wrapper around httpx. Another valid solution would be to use a GraphQL client library.
The entry point of the module is the ZeeneaGraphQLClient class.
This module provide a small scim 2.0 client restricted to recommended Zeenea patterns primitives. It is based on scim2_client library and complete some unimplemented method.
The entry point of the module is the ZeeneaScimClient class.
This module use dynaconf to load the documentation from setting files and environment variables.
See the read_configuration method for an example.
If you want to use dynaconf for your own project, read their documentation.
For now, it just provides create_parent: a function to create the parent folders of an output file if required.
All assets and code are under the CC0 LICENSE and in the public domain unless specified otherwise.