SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

🚀 Drive AI collaboration from Day 0 with Context Hive, a methodology that integrates AI as a key team member from the project's inception.

Big Data essentials: Hadoop, MapReduce, Spark. Explore tutorials and demos in Jupyter notebooks—most are self-contained and live, ready to run with a click.

My end-to-end Data Engineering learning journey. Includes concepts, notes, and real-world data pipeline projects. Based on hands-on experience with modern Big Data tools.

Docker image builds for Hadoop sandbox.

ETL Pipeline for Spar Nord Bank for the analysis of refilling frequency of the ATM's all over the europe

The Patient Alert ETL 🚑 project creates a real-time data pipeline to monitor vital health parameters from IoT devices in hospitals. Using Apache Kafka, Spark, and HBase, it processes streaming data and sends immediate alerts via Amazon SNS when vitals exceed normal thresholds, enhancing patient care through timely interventions.

Implementation of a big data management and analysis backbone architecture using PySpark for distributed and scalable data ingestion and MLlib for machine learning analysis. Part of Big Data Management and Analytics (BDMA) program.

Docker setup for Hadoop Single Node Cluster

This repository demonstrates big data processing, visualization, and machine learning using tools such as Hadoop, Spark, Kafka, and Python.

More than 2000+ Data engineer interview questions.

BD-Project: End-to-end Big Data infrastructure and processing for NBA stats. Includes HDFS architecture, comparative analysis of distributed computing (Spark vs. Hadoop), and RDD implementation for sports performance metrics.

100 SQL Practical Use Cases_Data Engineering

Proceso ETL

Architecture Lambda distribuée pour l'analyse en temps réel du trafic urbain et de la qualité de l'air dans les Smart Cities.

A complete Lambda Architecture pipeline for real-time and historical analysis of global energy consumption

Project done for my master's thesis. Collects research achievements from universities that are part of EUNICoast alliance, using Big Data tools, such as Spark

Hadoop and Hive fundamental commands

A fully-functional Hadoop Yarn cluster as docker-compose deployment.