This project provides a complete Data Engineering architecture to extract data from the Stripe API and process it through different database layers for transactional and analytical purposes.

This platform is composed of four core services, each in its own repository. The data flows from transactional processing (OLTP) to a final analytical warehouse (OLAP), managed by a central orchestrator.

• ➡️ oltp (Online Transaction Processing)

  • This service is responsible for capturing and storing raw data from the Stripe API. It likely handles real-time transactions, webhooks, and serves as the primary operational database.

• ➡️ nosql (NoSQL Database)

  • This component likely stores semi-structured or unstructured data, such as logs, customer interaction events, or JSON objects from the Stripe API, providing flexible and scalable storage.

• ➡️ olap (Online Analytical Processing)

  • This is the data warehouse of the project. Data from OLTP and NoSQL sources is transformed and loaded here to build a model optimized for business intelligence, reporting, and complex analytical queries.

• ➡️ orchestrator (ETL Orchestrator)

  • The brain of the data pipeline. This service (likely using a tool like Airflow) manages, schedules, and monitors the ETL/ELT jobs that move and transform data between the oltp, nosql, and olap systems.