When managing a large Jibri deployment, you usually want to autoscale using a strategy like "always keep N spare (not recording or livestreaming) jibri pods". This is difficult to achieve with the standard Kubernetes horizontal pod autoscaler (HPA).
jibri-pod-controller can be used as part of an alternative approach:
- Deploy
jibriusing a Deployment. Set the Deployment'sreplicasto the number of sparejibripods you want to run. - Deploy
jibri-pod-controllerin your cluster and give it RBAC permission to get/list/patchjibripods. - Configure
jibriin single use mode, and configure it to send webhook requests tojibri-pod-controller. - When a
jibripod starts to record or live-stream,jibri-pod-controllerwill patch the pod's labels so that they don't match the Deployment's label selector. This isolates thejibripod from the Deployment — the Deployment will immediately launch anotherjibripod to replace it (thus keeping the required number of spare pods), and the isolatedjibripod will continue to run. - When
jibrifinishes recording or live-streaming,jibri-pod-controllerwill delete the pod. A sweeper runs on a configurable interval to remove any expiredjibripods in casejibrifails to send the webhook for any reason. If multiplejibri-pod-controllerpods are running, one is elected to run the sweeper.
git clone https://github.com/avstack/jibri-pod-controller.git
cd jibri-pod-controller
docker build .
Irrelevant settings have been left out.
jibri {
id = "$POD_NAME"
single-use-mode = true
api {
http {
// This must match the JIBRI_HEALTH_PORT environment variable in the jibri-pod-controller deployment
external-api-port = 8080
}
}
webhook {
subscribers = [
// $POD_NAME must be substituted with the Jibri pod name, for example by using envFrom to set it as
// an environment variable and then using envsubst on jibri.conf in the container entrypoint script.
"http://jibri-pod-controller.default.svc.cluster.local/webhook/$POD_NAME"
]
}
}
- Replace the
imagewith the URL to your built container image ofjibri-pod-controller. - In this example, the
JIBRI_BUSY_LABELSare set toapp=jibri,state=busy. You could set up yourjibriDeployment to selectapp=jibri,state=idle.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jibri-pod-controller
namespace: default
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- list
- patch
- get
- delete
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- coordination.k8s.io
resourceNames:
- jibri-pod-controller-lease
resources:
- leases
verbs:
- update
- patch
- get
---
apiVersion: v1
kind: ServiceAccount
automountServiceAccountToken: true
metadata:
name: jibri-pod-controller
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: jibri-pod-controller
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jibri-pod-controller
subjects:
- kind: ServiceAccount
name: jibri-pod-controller
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: jibri-pod-controller
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: jibri-pod-controller
template:
metadata:
labels:
app: jibri-pod-controller
spec:
automountServiceAccountToken: true
serviceAccountName: jibri-pod-controller
terminationGracePeriodSeconds: 30
containers:
- name: jibri-pod-controller
image: your.image.registry.url/jibri-pod-controller:your-tag
env:
- name: RUST_LOG
value: info
- name: PORT
value: "8080"
- name: JIBRI_HEALTH_PORT
value: "8080"
- name: JIBRI_BUSY_LABELS
value: app=jibri,state=busy
- name: SWEEP_INTERVAL
value: "300"
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 8080
name: http
protocol: TCP
readinessProbe:
failureThreshold: 1
httpGet:
path: /
port: http
scheme: HTTP
initialDelaySeconds: 3
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 5m
memory: 8Mi
securityContext:
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
securityContext:
runAsGroup: 1000
runAsNonRoot: true
runAsUser: 1000
topologySpreadConstraints:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- jibri-pod-controller
maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- jibri-pod-controller
maxSkew: 1
topologyKey: failure-domain.beta.kubernetes.io/zone
whenUnsatisfiable: ScheduleAnyway
---
apiVersion: v1
kind: Service
metadata:
name: jibri-pod-controller
namespace: default
spec:
type: ClusterIP
selector:
app: jibri-pod-controller
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
jibri-pod-controller is licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Any kinds of contributions are welcome as a pull request.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in these crates by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
jibri-pod-controller development is sponsored by AVStack. We provide globally-distributed, scalable, managed Jitsi Meet backends.