Adapters facilitate the ingestion of data from various sources into TN, leveraging the SDK and Prefect for efficient data management.

Note: While adapters use a Python Framework, it's also possible to push data from other languages like Go or JavaScript. Refer to the respective SDKs for more information:

• Golang SDK
• JavaScript SDK

• Programming Knowledge:
  • Proficiency in Python
  • Prefect knowledge (https://www.prefect.io)
• Tools:
  • Docker
  • Prefect
  • Access to a TN Node (local or remote). You can refer to documentation here. TLDR: you can run task single:start from that repository after you clone it.

1. Clone the Repository:

git clone https://github.com/trufnetwork/adapters.git
cd adapters

2. Configure Environment Variables:

• Duplicate the .env.example file and rename it to .env.
• Update the environment variables in the .env file as needed.

3. Setup Virtual Environment:

We recommend using a virtual environment to manage dependencies. You can create one using the following commands:

python -m venv .venv
source .venv/bin/activate
pip install -e ."[dev]"

4. Launch Services with Docker Compose:

docker compose up -d

This command initializes the necessary services in the background.

5. Browser:

If you prefer to see the flows and blocks in browser, you can access http://localhost:4200/dashboard, on your local machine.

The repository offers several reusable tasks to facilitate data ingestion into TN. However you can develop your own tasks to fit your use-case, these tasks can serve as example how to implement custom adapter to your data:

Reading Data:

• read_repo_csv_file: Read a CSV file from a GitHub repository.
• task_read_gsheet: Retrieves data from a Google Sheet.

TN Operations:

• task_insert_tsn_records: Inserts records into TN.
• task_get_all_tsn_records: Fetches all records from TN.
• task_deploy_primitive_if_needed: Deploys a primitive if it doesn't already exist.

Data Manipulation:

• task_reconcile_data: Reconciles data between sources.
• task_normalize_source: Standardizes source data.
• task_filter_by_source_id: Filters data based on a source ID.
• task_prepare_records_for_tsn: Prepares records for insertion into TN.

The repository includes examples demonstrating data ingestion from Google Sheets:

• Direct Method: Specify the sheet ID and source ID directly.
• Dynamic Method: Retrieve the sheet ID and source ID from a CSV file in a GitHub repository.

Refer to the examples directory for detailed implementations.

• Python SDK
• Golang SDK
• JavaScript SDK
• Prefect Documentation