I've created this project for learning purposes and I keep running different implementations and experiments on it 🙂.
This project uses:
- Kubernetes with Helm Charts to run all services.
- Skaffold for development to support hot reloading with Kubernetes.
- Hashicorp Vault as the external secrets store.
- External Secrets Operator to dynamically create Kubernetes Secrets from external secret store (Vault).
After the project setup is ready, run the below command to start the local dev server.
skaffold dev --no-prune=false --cache-artifacts=falseTo start the kubernetes dashboard, run:
kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard-kong-proxy 8443:443Run the below command to build production images for the services.
docker compose -f compose.prod.yaml buildAnd then run the below command to install the helm chart with production config:
helm install pariksha ./helm -f ./helm/values.prod.yaml -n pariksha --create-namespaceNote: The auth service, mail service, and the Vault live outside the cluster and are not included in this repository.
I am using Docker Desktop Kubernetes for local development. But you may use any other local Kubernetes solution that suits your environment or preferences.
- Follow this official guide to install Docker Desktop.
- Follow this official guide to enable Kubernetes in Docker Desktop.
- Follow this official guide to setup Kubernetes Dashboard on your local.
- Follow this official guide to install Helm.
- Follow this official guide to install Skaffold.
- Install ESO using Helm by following this official guide.
This project uses Kubernetes Auth for Vault.
-
Prepare a Vault instance on your local.
-
Enable the Kubernetes Auth method.
vault auth enable -path=pariksha-kubernetes kubernetes- Run the below command at project root.
helm install pariksha-vault-auth ./vault --namespace pariksha --create-namespace- This Helm Chart will create a Service Account for Token Review.
- Copy the token from the Secret bound to this Service Account.
- This will be used as the
token_reviewer_jwtin step 6.
- Get the
kubernetes_hostusing the below command. This will be used in step 6.
kubectl cluster-infoFor Docker Desktop Kubernetes it will be something like "https://kubernetes.docker.internal:6443".
- Get the
kubernetes_ca_certusing the below command. It will be used in step 6.
kubectl get configmap kube-root-ca.crt -n kube-system -o jsonpath="{.data.ca\.crt}"- Configure the Kubernetes Auth Method uisng the values obtained from step 4, 5, and 6.
vault write auth/pariksha-kubernetes/config \
kubernetes_host="<your local kubernetes host>" \
token_reviewer_jwt="<your reviewer service account JWT>" \
kubernetes_ca_cert=@<path/to/ca.crt>Create the pariksha_policy.hcl file:
path "pariksha/data/engine" {
capabilities = ["read", "list"]
}
path "pariksha/data/exam" {
capabilities = ["read", "list"]
}
path "pariksha/data/paper" {
capabilities = ["read", "list"]
}
path "pariksha/data/question" {
capabilities = ["read", "list"]
}Write the Policy to Vault.
vault policy write pariksha_policy pariksha_policy.hclvault write auth/pariksha-kubernetes/role/external-secrets \
bound_service_account_names="external-secrets" \
bound_service_account_namespaces="external-secrets" \
policies="pariksha_policy"Enable a KV (Key-Value) secrets engine at the path pariksha.
vault secrets enable -path=pariksha -version=2 kv