A reference-grade, multi-cloud DevSecOps laboratory — secure-by-default infrastructure, signed software supply chain, GitOps delivery, and first-class AI/LLM security.
- Overview
- Architecture
- Tooling Matrix
- Repository Layout
- Prerequisites
- Quick Start
- Deployment Walkthrough
- AI Security (LLM Guardrails)
- Compliance & Threat Modeling
- Observability
- Contributing
- Security Policy
- License
My-DevSecOps-Lab is an opinionated, end-to-end blueprint demonstrating how to operate modern cloud-native and AI workloads under a shift-left + shield-right security model. It blends:
- Infrastructure as Code (Terraform) for AWS and GCP, with policy-as-code gates.
- GitOps Continuous Delivery for Kubernetes (Helm + Kustomize + Argo CD).
- Software Supply Chain Security — SBOMs, signed artifacts, provenance attestations (SLSA L3).
- Runtime Security — Falco, Tetragon, Kyverno admission control, default-deny networking.
- AI/LLM Security — prompt injection defenses, model supply chain integrity, OWASP LLM Top 10 red-teaming.
- Continuous Compliance — CIS, NIST 800-53, PCI-DSS, SOC 2 control mappings.
The lab is deliberately modular: every layer can be adopted in isolation or composed into a complete platform.
flowchart LR
subgraph Dev[Developer Workstation]
A[Code + pre-commit hooks]
end
subgraph CI[GitHub Actions - CI]
B[SAST / SCA / Secrets]
C[Container Build + Trivy]
D[SBOM - Syft]
E[Cosign keyless sign + SLSA attest]
end
subgraph Registry[OCI Registry]
F[(Signed Image + SBOM + Provenance)]
end
subgraph CD[Argo CD - GitOps]
G[App-of-Apps]
H[Kyverno verifies signature]
end
subgraph K8s[EKS / GKE]
I[Workloads]
J[Falco / Tetragon runtime]
K[LLM Gateway + Guardrails]
end
subgraph Cloud[AWS / GCP]
L[Terraform-managed VPC, IAM, KMS]
M[GuardDuty / Security Command Center]
end
A --> B --> C --> D --> E --> F
F --> H --> G --> I
L --> K8s
I --> J
I --> K
M --> Cloud
Design tenets
- Zero implicit trust — every artifact is signed, every policy is explicit, every network flow is denied by default.
- Reproducibility — pinned tool versions, hermetic builds, deterministic Terraform plans.
- Auditability — SBOMs, provenance, and policy decisions are persisted and queryable.
- AI-aware — LLM workloads are treated as a first-class threat surface (MITRE ATLAS + OWASP LLM Top 10).
| Domain | Tool | Purpose |
|---|---|---|
| IaC | Terraform | Multi-cloud provisioning (AWS + GCP) |
| IaC Security | tfsec, Checkov, Terrascan | Static analysis of Terraform |
| Policy as Code | OPA / Conftest, Sentinel | Pre-apply policy gates |
| Container Build | Docker BuildKit, ko | Reproducible image builds |
| Vulnerability Scan | Trivy, Grype | Image, FS, IaC, K8s scanning |
| SBOM | Syft, CycloneDX | SPDX/CycloneDX bills of materials |
| Signing | Cosign (Sigstore) | Keyless signing + verification |
| Provenance | SLSA GitHub generator | Build provenance attestations |
| SAST | Semgrep, CodeQL | Source code analysis |
| DAST | OWASP ZAP | Dynamic API/web scanning |
| Secrets | Gitleaks, TruffleHog | Secret detection (pre-commit + CI) |
| K8s Admission | Kyverno, Gatekeeper | Cluster policy enforcement |
| Runtime | Falco, Tetragon | eBPF-based runtime security |
| GitOps | Argo CD | Declarative continuous delivery |
| Secrets Mgmt | External Secrets Operator, HashiCorp Vault | Dynamic secret injection |
| Cloud Posture | AWS GuardDuty + Security Hub, GCP Security Command Center | CSPM / threat detection |
| Observability | Prometheus, Grafana, Loki, Tempo, OpenTelemetry | Metrics, logs, traces |
| AI Guardrails | NVIDIA NeMo Guardrails, Llama Guard, Microsoft Presidio | LLM input/output filtering |
| AI Red Team | garak, PyRIT | Adversarial LLM testing |
| Model Supply Chain | model-transparency, ModelScan | Model signing + malicious model detection |
.
├── .github/workflows/ # CI/CD pipelines (build, scan, sign, deploy, LLM red-team)
├── terraform/ # AWS + GCP modules, environments, OPA policies
├── kubernetes/ # Helm charts, Kustomize overlays, Argo CD apps, Kyverno policies
├── ai-security/ # LLM guardrails, red-team suites, model supply chain
├── ci-cd/ # Reusable GitHub composite actions and scripts
├── security/ # SAST/DAST/SCA configs, signing keys, compliance mappings
├── observability/ # Prometheus, Grafana, Loki, Tempo, OTel configs
├── docs/ # Architecture, ADRs, runbooks, threat models
└── scripts/ # Bootstrap and teardown helpers
See each subdirectory's README.md for module-level details.
| Tool | Version | Notes |
|---|---|---|
| Terraform | ≥ 1.7 | Use tenv or tfenv for pinning |
| kubectl | ≥ 1.29 | |
| Helm | ≥ 3.14 | |
| Kustomize | ≥ 5.3 | |
| Argo CD CLI | ≥ 2.10 | |
| Cosign | ≥ 2.2 | Keyless signing via OIDC |
| Trivy | ≥ 0.50 | |
| Syft | ≥ 1.0 | |
| Docker / Podman | recent | BuildKit enabled |
| pre-commit | ≥ 3.6 | pre-commit install after clone |
| Cloud CLIs | aws ≥ 2.15, gcloud ≥ 470 | For OIDC + state backends |
Cloud accounts — an AWS account with admin-equivalent role for bootstrap, and/or a GCP project with Owner for the lab environment. Production deployments must use a delegated, scoped role.
# 1. Clone and install hooks
git clone https://github.com/<owner>/My-DevSecOps-Lab.git
cd My-DevSecOps-Lab
pre-commit install
# 2. Spin up a local kind cluster with policies pre-installed
make local-up
# 3. Deploy the sample app + LLM gateway via Argo CD
make argocd-bootstrap
# 4. Run the full security gate locally
make scan-allcd terraform/environments/dev/aws
terraform init -backend-config=backend.hcl
terraform plan -out=plan.tfplan
conftest test plan.tfplan -p ../../../policies/opa # OPA gate
tfsec . && checkov -d . # static checks
terraform apply plan.tfplanThe eks-secure module provisions a private EKS cluster with IRSA, KMS-encrypted secrets, audit logging to CloudWatch, and a default-deny security group baseline. The github-oidc module wires GitHub Actions to AWS without long-lived credentials.
aws eks update-kubeconfig --name devsecops-lab-dev
kubectl apply -k kubernetes/policies/kyverno/
kubectl apply -f kubernetes/security/network-policies/default-deny.yaml
helm upgrade --install argocd argo/argo-cd -n argocd --create-namespace
kubectl apply -f kubernetes/argocd/bootstrap/Kyverno enforces:
require-signed-images— only Cosign-verified images may run.disallow-privileged— no privileged pods, no host namespaces.require-resource-limits— every pod declares CPU/memory limits.
The ci-build-scan.yml workflow runs on every PR:
- SAST — Semgrep + CodeQL.
- SCA — Trivy filesystem + OSV Scanner.
- Secrets — Gitleaks.
- Build — reproducible BuildKit image.
- Image Scan — Trivy (fail on HIGH/CRITICAL with fixed version).
- SBOM — Syft → SPDX + CycloneDX.
- Sign — Cosign keyless (GitHub OIDC → Fulcio → Rekor).
- Provenance — SLSA Level 3 generator publishes attestation.
- Push — to GHCR with
:sha-<commit>and:latesttags.
Argo CD reconciles the kubernetes/argocd/applications/ manifests against the cluster. Image updates are gated by Kyverno's signature verification policy — unsigned or unattested images are rejected at admission.
cosign verify ghcr.io/<owner>/sample-app:<sha> \
--certificate-identity-regexp 'https://github.com/<owner>/.+' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
cosign verify-attestation --type slsaprovenance ghcr.io/<owner>/sample-app:<sha>The ai-security/ tree is the lab's most distinctive component. It treats LLM-backed services as a dedicated attack surface with its own pipeline.
User Prompt
│
▼
┌─────────────────────────────────────────┐
│ Input Filters (ai-security/guardrails) │
│ • Prompt-injection classifier │
│ • PII redaction (Presidio) │
│ • Jailbreak heuristics │
└─────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ LLM Gateway (policies + rate limits) │
│ • NeMo Guardrails dialog policies │
│ • Tenant isolation │
│ • Audit log → SIEM │
└─────────────────────────────────────────┘
│
▼
Model
│
▼
┌─────────────────────────────────────────┐
│ Output Filters │
│ • Llama Guard / toxicity │
│ • Secret-leakage regex + entropy │
│ • Hallucination grounding check │
└─────────────────────────────────────────┘
│
▼
Response
Every PR that touches ai-security/** triggers llm-redteam.yml:
- garak runs the OWASP LLM Top 10 probe set against the staging gateway.
- PyRIT scenarios exercise multi-turn jailbreaks and exfiltration.
- Results are uploaded as SARIF → GitHub Code Scanning.
- AI-BOM generated for every model (training data sources, licenses, fine-tunes).
- Model signing via sigstore/model-transparency.
- ModelScan detects malicious pickle payloads before models reach production.
security/compliance/ provides control-mapping spreadsheets and evidence collectors for:
- CIS Benchmarks — Kubernetes, EKS, GKE, Docker.
- NIST 800-53 Rev. 5 — moderate baseline.
- PCI-DSS v4.0 — relevant requirements for cardholder workloads.
- SOC 2 — Trust Service Criteria mapping (CC6, CC7, CC8).
Threat models follow STRIDE and are versioned alongside the components they describe — see docs/threat-models/.
A unified telemetry stack is provided under observability/:
- Metrics — Prometheus + recording rules.
- Logs — Loki with structured pipeline stages.
- Traces — Tempo + OpenTelemetry Collector.
- Dashboards — pre-built Grafana dashboards for cluster, application, and AI safety signals (refusal rate, guardrail trip count, prompt-injection blocks, latency per model).
See CONTRIBUTING.md. All commits must be signed (git commit -S) and pass the pre-commit gate.
See SECURITY.md for the responsible-disclosure process. Do not open public issues for vulnerabilities.
Distributed under the Apache License 2.0.
Lab disclaimer — this repository is a learning and demonstration platform. Review every module against your own threat model before adopting it in production.