A minimal KFP v2 (Kubeflow) pipeline compiled to JSON and executed on Google Vertex AI Pipelines. Includes both GUI and Linux CLI flows.
- generate_numbers → writes CSV with numbers 0..count-1
- sum_numbers → reads CSV and returns the sum
- count is a runtime parameter (choose in GUI or pass via CLI)
- gcloud installed and authenticated
- A Google Cloud project with billing enabled
- Python 3.10+
- A GCS bucket for pipeline artifacts (pipeline root)
- pipelines/simple_pipeline.py — pipeline definition
- simple_pipeline.json — compiled PipelineSpec (created at compile step)
# Set project and region
export PROJECT_ID="YOUR_PROJECT_ID"
export REGION="YOUR_REGION"
gcloud auth login
gcloud config set project "$PROJECT_ID"
gcloud config set ai/region "$REGION"
# Enable required APIs
gcloud services enable aiplatform.googleapis.com storage.googleapis.com
# Create a GCS bucket (change name if it already exists)
export BUCKET="${PROJECT_ID}-vertex-pipelines"
gsutil mb -p "$PROJECT_ID" -l "$REGION" "gs://${BUCKET}"Optional: if you hit permissions on the bucket, grant Vertex AI service agent access.
PROJECT_NUMBER="$(gcloud projects describe "$PROJECT_ID" --format='value(projectNumber)')"
SA="service-${PROJECT_NUMBER}@gcp-sa-aiplatform.iam.gserviceaccount.com"
gsutil iam ch "serviceAccount:${SA}:roles/storage.objectAdmin" "gs://${BUCKET}"# Create venv and install deps
cd vertex-pipeline-demo
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip kfp google-cloud-pipeline-components google-cloud-aiplatform pandas
# Compile
python pipelines/simple_pipeline.py
# Output: simple_pipeline.json-
Open Vertex AI Pipelines in Console: https://console.cloud.google.com/vertex-ai/pipelines?project=
-
Click + Create run (or + New, then upload pipeline spec).
- Upload simple_pipeline.json
- Pipeline root: gs:///pipeline-root
- Parameters: set count (e.g., 10)
-
Start the run, watch the DAG, and view outputs.
- The pipeline returns a parameter “sum” visible in the run’s outputs.
export JOB="simple-pipeline-$(date +%s)"
gcloud ai pipeline-jobs submit "$JOB" \
--region="$REGION" \
--pipeline-spec-file="simple_pipeline.json" \
--pipeline-root="gs://${BUCKET}/pipeline-root" \
--parameter-values=count=10
# Check status
gcloud ai pipeline-jobs describe "$JOB" --region="$REGION"
gcloud ai pipeline-jobs list --region="$REGION"Change count to any integer at submit time.
- Components install pandas at runtime via packages_to_install (no custom image required).
- The pipeline’s output “sum” is returned from sum_numbers and exposed at the pipeline level for easy viewing in the Console.