systemd: What to Know and
How to Transition
Tom Sorensen
Solutions Architect, Red Hat
2 RED HAT | Tom Sorensen
Agenda
What is systemd?
What does systemd offer?
How does this affect me?
How do I transition to systemd?
Whats the next step?
4 RED HAT | Tom Sorensen
What is systemd?
5 RED HAT | Tom Sorensen
What is systemd?
systemd is a suite of system management daemons,
libraries, and utilities designed as a central management
and configuration platform for the Linux computer
operating system.
- From http://en.wikipedia.org/wiki/Systemd
6 RED HAT | Tom Sorensen
*YAWN*
7 RED HAT | Tom Sorensen
What is systemd?
Replaces init literally
init is now a symlink to systemd
First process to start and last to stop
Parent process of all other processes
Manages services and other resources
9 RED HAT | Tom Sorensen
But I liked init...
init is really, really old
systemd adopted across all major Linux versions
Default in Fedora 15 as
of May 2011
Slackware in the
enterprise anyone?
10 RED HAT | Tom Sorensen
What does systemd offer?
11 RED HAT | Tom Sorensen
What does systemd offer?
Controls More than Services
Dependency Control
Service Activation
Faster Start Up and Shutdown
Improved Resource Management
Tracks and Restarts Services
Improved Logging, Debugging and Profiling
Plenty More...
12 RED HAT | Tom Sorensen
Units
Controls more than services, it controls all resources on
the system - referred to as units.
Examples of Units:
Services Sockets
Targets and more..
Units are defined using Unit Files
Naming convention is name.unit_type
13 RED HAT | Tom Sorensen
Dependencies
Since systemd controls more than services, it can also
more intelligently handle dependencies.
Define order and requirements for each unit
Example: nfs-lock.service
Requires=rpcbind.service network.target
After=network.target named.service rpcbind.service
Before=remote-fs-pre.target
No more semi-arbitrary 00-99 ASCII order loading
14 RED HAT | Tom Sorensen
Common Unit File Options
Description=Unit description
Documentation=Documentation links
Requires=Additional units required
Before/After=Unit must start Before/After
Wants=Weaker Requires
Conflicts=Units cannot co-exist
WantedBy/RequiredBy=Set other units requirement
Lots of great detail in the RHEL 7 System Administrator's Guide
15 RED HAT | Tom Sorensen
Service Activation
With dependency control for all resources, systemd can
also activate services on demand.
Start up services when needed
Save resources
Increased reliability
Transparent to client
Activation by Socket, Device, Path, Bus, and Timer
Recommended to convert xinetd services to units
16 RED HAT | Tom Sorensen
Parallel, Not Serial
With defined dependencies we dont need to wait for
other services, sockets, file systems, etc that arent
needed for that specific unit.
Allows for Faster Start Up and Shutdown
Why does this matter?
Downtime matters Five 9s = 5.26 minutes per year
Cloud - Charges by runtime
Capacity on demand
17 RED HAT | Tom Sorensen
Improved Resource Management
Services are labeled with individual cgroups, including all
components and children. Resources are also divided
into Slices.
Services labeled and isolated with Cgroups
More control than nice alone
Can configure multiple instances for a single service
Can balance by shares or by hard limits
18 RED HAT | Tom Sorensen
Kill/Restart Cleanly
With each service being tracked by individual cgroup,
systemd has an understanding of all related processes.
Tracked in the kernel
Knows all children
Dont need to rely on a potentially misbehaving
process to hopefully kill its children
19 RED HAT | Tom Sorensen
Auto-Restarting
With more control over the whole service chain, systemd
can track the state of all services and make corrections
with less impact.
Its paying attention!
Reality: software does crash occasionally
Reduces need for manual intervention
Socket stays open, only lose that single transaction
20 RED HAT | Tom Sorensen
Improved Logging
Every service logs, from when the initial RAM disk is
loaded to start the kernel to final shutdown of the system,
from all channels.
Dont need to wait for syslog to start
No More Losing STDERR and STDOUT
More detail than classic syslog alone
Improved debugging and profiling
21 RED HAT | Tom Sorensen
(This is the end of the overview, well now get to the practical.)
22 RED HAT | Tom Sorensen
How does this affect me?
23 RED HAT | Tom Sorensen
How does this affect me?
File Locations
Targets vs Runlevels
Booting with systemd
Analyzing the Boot Process
Resetting the Root Password
Systemctl and Other Commands
Logging and journalctl
24 RED HAT | Tom Sorensen
File Locations
Maintainer files: /usr/lib/systemd/system/
Administrator files: /etc/systemd/system/
Drop-in configs: /etc/systemd/system/<name.unit>.d/
Non-persistent, runtime data: /run/systemd/
Systemd journal, non-persistent: /run/log/journal
Note: unit files under /etc will take precedence over /usr
25 RED HAT | Tom Sorensen
Targets are the new Runlevels
Targets are logical groupings of services and other units
that are started and stopped together.
Targets != Runlevels some equivalency
Traditional Runlevel Equivalent Target Symlink Target
Runlevel 0 poweroff.target runlevel0.target
Runlevel 1 rescue.target runlevel1.target
Runlevel 2 multi-user.target runlevel2.target
Runlevel 3 multi-user.target runlevel3.target
Runlevel 4 multi-user.target runlevel4.target
Runlevel 5 graphical.target runlevel5.target
Runlevel 6 reboot.target runlevel6.target
Targets can and will contain other targets
26 RED HAT | Tom Sorensen
Common Targets
Common targets are similar to traditional runlevels.
Target Purpose
graphical.target Supports multiple users, graphical and text-based logins
multi-user.target Supports multiple users, text-based logins only
rescue.target Single user, local file systems mounted and basic system
initialization completed, networking is not activated
emergency.target Single user, root file system is mounted read-only, only a few
essential services are started, networking is not activated
Rescue and Emergency require root password!
27 RED HAT | Tom Sorensen
Working with Targets
Viewing the default target:
Setting default target:
Default target is just a symlink:
28 RED HAT | Tom Sorensen
Working with Targets
Changing currently loaded target:
Changing to rescue mode:
Changing to emergency mode without sending message:
29 RED HAT | Tom Sorensen
Working with Targets
View list of currently loaded targets:
Results pipe to less by default: (can use no-pager)
Not a single runlevel, but a a collection of targets
30 RED HAT | Tom Sorensen
Shutting Down, Suspending, Etc.
Old Command New Command Description
halt systemctl halt Halts the system
poweroff systemctl poweroff Powers off the system
reboot systemctl reboot Restarts the system
pm-suspend systemctl suspend Suspends the system
pm-hibernate systemctl hibernate Hibernates the system
pm-suspend-hybrid systemctl hybrid-sleep Hibernates and
suspends the system
31 RED HAT | Tom Sorensen
Boot Process Quick Review
BIOS/UEFI
Bootloader
Kernel and Initrd
System Initialization
32 RED HAT | Tom Sorensen
Boot Process
Boot path determined by default.target
Lets track it backwards!
33 RED HAT | Tom Sorensen
Boot Process
We see graphical.target requires multi-user.target and
should start after it. Lets look at multi-user.target...
Next, we see multi-user.target requires basic.target...
34 RED HAT | Tom Sorensen
Boot Process
Which requires basic.target...
Which requires sysinit.target...
No more requires, but still some wants...
35 RED HAT | Tom Sorensen
Boot Process
Which wants local-fs-pre.target and swap.target...
Once we look in these we see weve reached the end of
the line!
36 RED HAT | Tom Sorensen
Boot Process
Once systemd has the beginning of the chain, it starts
processing at the earliest step, starting all of the
associated units in the specified order.
But, how does this work for starting individual services?
37 RED HAT | Tom Sorensen
Boot Process Services/Units
Each target may also have a wants directory in addition
to the specific Wants and Requires directives in the
unit files.
Target Wants Directories:
/usr/lib/systemd/system/<name>.target.wants/
/etc/systemd/system/<name>.target.wants/
Files are symlinks to actual unit files
38 RED HAT | Tom Sorensen
Boot Process - Services/Units
Example for multi-user.target.wants:
/usr/lib are static units for the OS, /etc are system specific
39 RED HAT | Tom Sorensen
Analyzing Boot Process
Each unit is tracked during start up
Can see what is taking the longest to start at boot
40 RED HAT | Tom Sorensen
Analyzing Boot Process
Analysis data can also be exported as SVG
41 RED HAT | Tom Sorensen
Resetting Root Password
With the change from runlevels to targets, we also have a
change to the traditional single user mode.
Rescue and emergency targets require root password
Lets not learn this at 3am during a fire!
Resetting the Root Password of RHEL-7 / systemd
https://access.redhat.com/solutions/918283
42 RED HAT | Tom Sorensen
Resetting Root Password
Step 1: Reboot your system
Step 2: When the GRUB2 boot loader menu appears,
press any key to interrupt, and press e to edit selection
43 RED HAT | Tom Sorensen
Resetting Root Password
Step 3: Using the cursor keys, navigate to the end of the
line that starts with linux16
Step 4: Append rd.break to the end of the line and press
Ctrl-X to boot modified config
44 RED HAT | Tom Sorensen
Resetting Root Password
Step 6: At the switch_root prompt, remount the /sysroot
file system read-write, then use chroot to enter the jail
Step 7: Reset the root password with passwd
45 RED HAT | Tom Sorensen
Resetting Root Password
IMPORTANT! (Because youre using SELinux, right?)
Step 8: Instruct SELinux to relabel all files upon reboot.
System may not boot properly if skipped!
Step 9: Exit twice to initiate relabel and reboot
NOTE: The relabel may take quite a while to run before
the system becomes available.
46 RED HAT | Tom Sorensen
systemctl vs service
Start, status, stop a service:
47 RED HAT | Tom Sorensen
systemctl vs service
Note the different order of command option service
instead of command service option. This allows you to
run systemctl against multiple services at once.
Systemsctl start/stop will be silent unless there is an
error, in which case it will print to screen and log.
48 RED HAT | Tom Sorensen
systemctl vs service
List services:
49 RED HAT | Tom Sorensen
systemctl vs chkconfig
Enable, check, disable a service:
--list is not as important because you can see this in status
50 RED HAT | Tom Sorensen
systemctl vs chkconfig
List all services:
51 RED HAT | Tom Sorensen
systemctl
List all services by target as a dependency tree:
52 RED HAT | Tom Sorensen
systemctl
Lots of options...
53 RED HAT | Tom Sorensen
systemd-cgtop
Show top control groups by their resource usage:
May need to enable accounting perfect drop-in!
54 RED HAT | Tom Sorensen
systemd-cgls
Recursively show control group contents:
55 RED HAT | Tom Sorensen
systemd-*
Lots of new options...
56 RED HAT | Tom Sorensen
journalctl
View basic logs:
Time stamps converted to system local time zone
All logged data is shown, including rotated logs
Non-persistent by default, can be preserved
57 RED HAT | Tom Sorensen
journalctl
View most recent logs: (-f to follow)
Can force stdout/stderr to write to journal with
systemd-cat if wanted
59 RED HAT | Tom Sorensen
journalctl
Specify form:
Other options:
60 RED HAT | Tom Sorensen
journalctl
Filter by priority:
Filter by time and priority:
Advanced filtering by field, UID, unit, etc..
61 RED HAT | Tom Sorensen
Systemd Journal
How to enable persistent logging for the systemd journal
https://access.redhat.com/solutions/696893
System Administrator's Guide
https://access.redhat.com/documentation/en-US/Red_Hat_Enter
prise_Linux/7/html/System_Administrators_Guide/s1-Using_the
_Journal.html
Lennart Poettering - The systemd Journal
https://www.youtube.com/watch?v=i4CACB7paLc
62 RED HAT | Tom Sorensen
The future is not so future any more...
(RHEL 7 is here and systemd is part of every major Linux distro. Its time to start using it.)
63 RED HAT | Tom Sorensen
How do I transition to
systemd?
64 RED HAT | Tom Sorensen
How do I transition?
Start using the new commands
Learn more about cgroups
Convert your init scripts
Drop-in config overrides
Benefit from auto-restart
Address rc.local
Convert xinetd
65 RED HAT | Tom Sorensen
Start using the new commands
Bash Completion is your friend!
service
# yum install bash-completion
chkconfig
systemd Cheat Sheet for Red Hat Enterprise Linux 7
https://access.redhat.com/articles/systemd-cheat-sheet
Common Administrative Commands in RHEL 5, 6, & 7
https://access.redhat.com/articles/1189123
66 RED HAT | Tom Sorensen
Embrace cgroups
Already available in RHEL 6
More easily controlled in RHEL 7
Resource Management and Linux Containers Guide
https://access.redhat.com/documentation/en-US/Red_Hat_Enterpris
e_Linux/7/html/Resource_Management_and_Linux_Containers_Guide/i
ndex.html
67 RED HAT | Tom Sorensen
Convert your init scripts
How to convert init scripts to systemd units files
https://access.redhat.com/solutions/912263
Step 1: Check out your simple init script
68 RED HAT | Tom Sorensen
Convert your init scripts
Step 1, continued...
69 RED HAT | Tom Sorensen
Convert your init scripts
Step 1, continued a bit more...
70 RED HAT | Tom Sorensen
Convert your init scripts
Step 2: Modify/Install your RHEL 6 package on your
RHEL 7 system and ensure it works with a manual start
Step 3: Create a unit file on the RHEL 7 system
71 RED HAT | Tom Sorensen
Convert your init scripts
Step 4: Notify systemd of the new unit file
Step 5: Start the service and verify it is running
72 RED HAT | Tom Sorensen
Convert your init scripts
Step 6: Enabled the service to start on boot once youve
verified its functioning as expected
Step 7: Profit!
73 RED HAT | Tom Sorensen
Drop-in Config Overrides
Drop-in configs allows you to override only specific
directives without overriding the entire unit file.
If you just copy the maintainer file from /usr/lib into /etc
and make changes your system will use only the
directives in that /etc unit file. This means that if the
maintainers improve or change something in the default
unit file your system will not ever see the new changes.
74 RED HAT | Tom Sorensen
Drop-in Config Overrides
Step 1: Create /etc/systemd/system/<name.unit>.d/
directory
Step 2: Create a file containing only the directives youd
like to modify
Step 3: Reload systemd (systemd will warn you in status)
75 RED HAT | Tom Sorensen
Drop-in Config Overrides
Step 4: Restart your service and check the new status
76 RED HAT | Tom Sorensen
Auto-Restarting Processes
SSHD configured to auto-restart by default
If main process dies, it will restart after 42 seconds
77 RED HAT | Tom Sorensen
Using rc.local
rc.local is supported, but no longer runs last
chmod +x /etc/rc.d/rc.local
Just a service - /usr/lib/systemd/system/rc-local.service
78 RED HAT | Tom Sorensen
Convert xinetd
Example: Converting SSHD to Run On-Demand
Stolen from: http://0pointer.de/blog/projects/inetd.html
Step 1: Create an /etc/systemd/system/
[email protected]file like the following example
Note the @ after the sshd, this marks it as an instance
79 RED HAT | Tom Sorensen
Convert xinetd
Step 2: Disable sshd.service
Step 3: Enable sshd.socket
Probably a good idea to make sure you have non-SSH
access to this machine as well. ;)
80 RED HAT | Tom Sorensen
Convert xinetd
Step 4: Stop sshd.service and start sshd.socket
Step 5: Reconnect with SSH and check the new
connection
81 RED HAT | Tom Sorensen
Reaching Maximum Capacity!
82 RED HAT | Tom Sorensen
Review: systemd
Replaces init and does much more
It is here and its powerful
New boot and root password reset process
New commands and functionality
Plenty of great information and resources available
83 RED HAT | Tom Sorensen
Whats the next step?
84 RED HAT | Tom Sorensen
Continue planning your transition
Define your goals Is it time to embrace SELinux?
Continue your research
RHEL 7 Migration Planning Guide
https://access.redhat.com/documentation/en-US/Red_Hat_Ente
rprise_Linux/7/html/Migration_Planning_Guide/
Navigating Red Hat Enterprise Linux 7
https://access.redhat.com/articles/887923
Talk to your TAM and SA!
85 RED HAT | Tom Sorensen
Learn More
Lots of great info at https://access.redhat.com and
http://www.freedesktop.org/wiki/Software/systemd/
Overview of systemd for RHEL 7
https://access.redhat.com/articles/754933
Red Hat Summit 2013 - Getting Ready for systemd
https://access.redhat.com/videos/403833
RHEL 7 - systemd Service & Resource Management
https://access.redhat.com/videos/898503
Rethinking PID 1 by Lennart Poettering
http://0pointer.de/blog/projects/systemd.html
86 RED HAT | Tom Sorensen
Learn More not just Red Hat!
Arch Wiki
https://wiki.archlinux.org/index.php/Systemd
Arch Why We Switched
https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530
Gentoo Documentation
https://wiki.gentoo.org/wiki/Systemd
OpenSUSE, Debian, Ubuntu also on or transitioning to systemd
87 RED HAT | Tom Sorensen
Thank you!
88 RED HAT | Tom Sorensen
