MLOPs End-to-End Pipeline

Machine Learning Dev Ops Guideline

1a. AWS - EC2 - 1b. AWS - SageMaker - 1c. AWS - S3- 1c. AWS - ECR

or

1d. GCP Google Cloud Platform (Vertex AI)

2. Ubuntu Linux setup

3. Git

git init

4. DVC (Data Version Control)

Split Jupyter Notebook into Python Scripts (Modular Approach):

1. data_ingestion.py

2. data_preprocessing.py

3. feature_engineering.py

4. model_building.py

5. model_evaluation.py

6. Output metrics.json

Create dvc.yaml (configuration file)

git init & dvc init , dvc repro (this will execute dvc.yaml) , dvc dag , dvc metrics show

5. Experiment Tracking

MLflow & Dagshub

See here MLflow Repo

6. Docker

Dockerfile