Section Overview: Operating Swarm

• Service Logs
• Docker Events
• Swarm Configs
• Rolling Service Updates
• Healthchecks in Updates and Rollbacks
• Portainer Web UI
Section Requirements
• Created a 3-node (or more) Swarm with your environment of
choice
• Not all nodes are managers (mine is 1 manager and 2 workers)
• Created the Swarm Visualizer service from previous lecture
• Cleared other stacks/services/containers/volumes/networks
 Run The Swarm Visualizer
• Start a service for the Docker Swarm Visualizer
• This is a useful learning graphic that shows us how tasks move
around
• https://github.com/dockersamples/docker-swarm-visualizer
• docker service create \
--name=viz \
--publish=8080:8080/tcp \
--constraint=node.role==manager \
--mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
dockersamples/visualizer

docker service create --name=viz --publish=8080:8080/tcp --constraint=node.role==manager --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock dockersamples/visualizer

 Service Logs
• Same as docker container logs, but aggregates all service tasks
• Can return all tasks at once, or just one task's logs
• Great for real time cli troubleshooting
• Has options for follow/tail, and other common log options
• Not for log storage, or searching, or feeding into other systems
• Doesn't work if you use --log-driver for sending logs off server
• Therefore: use for small Swarms and testing

works with json-file or journald log drivers

json-file is default driver
 Service Log Examples
• Return all logs for a service
• docker service logs <servicename/id>
• Return all logs for a single task
• docker service logs <taskid>
• Return unformatted logs with no trunking
• docker service logs --raw --no-trunc <servicename/id>
• Only return last 50 log entries and all future logs
• docker service logs --tail 50 --follow <servicename/id>

cd swarm-stack-5
docker stack deploy -c example-voting-app-stack.yml vote
watch docker stack services vote (not on windows)

docker ps
docker exec $(docker ps --filter name=vote_vote -q) ./generate-some-votes.sh

docker service ls
docker service logs vote_worker
docker service ps vote_worker
docker service logs <task id>
docker service logs --tail 5 --follow vote_worker
docker service logs --raw --no-trunc vote_worker

grep logs (not on windows)

docker service logs vote_worker
docker service logs vote_worker 2>&1 | grep
docker service logs vote_worker 2>&1 | findstr
Lecture Cleanup
• Remove the stack we created

> docker stack rm vote

 Docker Events
• "Actions Taken" Logs of Docker Engine and Swarm
• e.g. "network create", "service update", "container start"
• docker events received SwarmKit events in 17.06
• services/nodes/secrets/configs got create/update/remove
• Has searching (filtering) and formatting
• Limited to last 1,000 Events (no logs are stored on disk)
• Two scopes, "local" and "swarm"
• Not the same as dockerd (journald) log, also not an error log

PR of cluster events: https://github.com/moby/moby/pull/32421

https://github.com/moby/moby/issues/29548#issuecomment-268329529

https://docs.docker.com/engine/reference/commandline/events/
 Docker Events Examples
• Follow future events
• docker events

• Return events from a date until now and future

• docker events --since 2017-12-01
• docker events --since 2017-12-01T12:30:00
• Return events from 30m ago until now and future
• docker events --since 30m
• docker events --since 2h10m

• Return last hour of events filtered by event name

• docker events --since 1h --filter event=start

• Only return Swarm related events for networks

• docker events --since 1h --filter scope=swarm --filter type=network

REFERENCES
https://docs.docker.com/engine/reference/commandline/events/

docker service scale viz=0

Setup three windows for three event monitors

docker service create --name nginx nginx

docker service rm nginx

Use stress example then look for "container oom" in events

docker service create --limit-memory 100M --name 100 bretfisher/stress:256m

docker service scale viz=1

 Swarm Configs
• Map files/strings stored in Raft Log to any file path in tasks
• Ideal For nginx/mysql/etc. configs
• Now you don't need custom image or bind-mount to host 😨
• 17.06+ similar to Secrets but can go anywhere in container
• Immutable, so rotation process is key
• Removable once services are removed
• Strings saved to Swarm Raft Log (instant HA)
• Private keys should still use Secrets (RAM disks, enc at rest)

REFERENCES
https://docs.docker.com/engine/swarm/configs/
 Swarm Config Examples
• Create a new Config from a nginx config
• docker config create nginx01 ./nginx.conf
• Creating a Service with a Config
• docker service create --config source=nginx01,target=/etc/nginx/conf.d/
default.conf
• Creating new Config to replace old
• docker config create nginx02 ./nginx.conf
• Updating Service with new Config
• docker service update --config-rm nginx01 --config-add source=nginx02,target=/etc/
nginx/conf.d/default.conf

docker stack deploy -c example-voting-app-stack.yml vote

cat nginx

docker config create vote-nginx-20171211 ./nginx-app.conf

cat example

docker service create --config source=vote-nginx-20171211,target=/etc/nginx/conf.d/default.conf -p 9000:80 --network vote_frontend --name proxy nginx

check it works on http://localhost:9000 and shows voting site

Inspect config

Try to remove used config

docker config rm vote-nginx-20171211

Inspect service with config

docker config create --label vote-nginx vote-nginx-20171212 ./nginx-app.conf

docker service update --config-rm vote-nginx-20171211 --config-add source= vote-nginx-20171212,target=/etc/nginx/conf.d/default.conf proxy

Swarm Configs in Stack Files
version: "3.3" #3.3 or higher required
services:
web:
image: nginx
configs:
- source: nginx-proxy
target: /etc/nginx/conf.d/default.conf
configs:
nginx-proxy:
file: ./nginx-app.conf
Lecture Cleanup
• Remove the stack, service and configs we created

> docker stack rm vote

> docker service rm proxy

> docker config rm vote-nginx-20171211 vote-nginx-20171212