Curs Linux 2017 PDF
Uploaded by
Valentin ToderiucCurs Linux 2017 PDF
Uploaded by
Valentin ToderiucLinux Administration
1. Installing the OS
© 2017 All rights reserved.
Structure of an Operating System
© 2017 All rights reserved.
Structure of an Operating System
User
You and me
Applications
OpenOffice Writer, Mozilla Firefox
Operating System
Kernel – Linux, GNU Hurd
Modules – pcnet32, cdrom, ip_nat, ext3
Shell – bash, sh, csh, zsh
Tools – cp, mv, rm
Hardware
CPU, Memory, GPU, HDD
© 2017 All rights reserved.
Structure of an Operating System
Kernel
Core of the system
Links hardware and software
First piece of code of the OS to be loaded into memory
© 2017 All rights reserved.
Structure of an Operating System
The Linux Kernel
Monolithic
Runs in kernel space (supervisor mode)
Offers an interface to the hardware via a set of primitives (system calls)
Non-critical software (e.g. GUI) does not run in the kernel, but in user space
© 2017 All rights reserved.
FYI – Types of OS Kernels
© 2017 All rights reserved.
Structure of an Operating System
Modules
Each module offers a service to the OS (e.g. memory management, process management)
Modules are dynamically loaded only the necessary modules are loaded at any given
time
Modules interact through the kernel, via system calls
© 2017 All rights reserved.
Structure of an Operating System
Shell
The interface between user/applications and the kernel
Can be a CLI (command line interface), or a GUI (graphical user interface)
Examples:
CLI: Bourne shell (sh), Bourne Again Shell (bash), Korn shell (ksh), C shell (csh)
GUI: Gnome, KDE, CDE, XFCE
© 2017 All rights reserved.
Structure of an Operating System
Tools
Small applications used for basic functions
cp, rm, mv
C compiler
Assembler
Used directly by the user (at the shell prompt), by more complex programs, or by the shell (shell
script)
The first tools were written by the GNU community.
Linux Kernel + modules + shell(s) + other software a GNU/Linux Distribution
© 2017 All rights reserved.
GNU/Linux Distributions
Distribution
Operating system (kernel, shell, tools)
Installer
Bootloader
Package manager
Applications – web browser, e-mail client, office suite
Hundreds of different distributions at the moment -
http://en.wikipedia.org/wiki/List_of_linux_distributions
© 2017 All rights reserved.
What is the BEST distribution?
The one that you are familiar with!
© 2017 All rights reserved.
Before Installation
Choose the right distribution based on:
Needs:
Workstation or server?
Stability, or cutting-edge features?
Home, or enterprise?
Hardware:
GUI, or CLI-only?
I386? PowerPC? X64?
Drivers?
Administrator (remember the “best distribution” rule!)
© 2017 All rights reserved.
Before Installation (2)
Choose the right installation type
CD/DVD
With or without a LiveCD
Network install
Update vs. fresh install
© 2017 All rights reserved.
Installation Steps
Basically the same for most distributions (not necessarily in the same order):
Choose language, keyboard, mouse
Create partitioning scheme
Configure bootloader
Select and install software packages
Configure root password and create users
Configure X
Configure network
Other configs (sound, firewall, automatic updates, etc.)
© 2017 All rights reserved.
Linux Administration
2. Working with the Shell
© 2017 All rights reserved.
Linux command basics
What is a command shell
Command shells
Command structure and syntax
Help and man pages
Navigation
© 2017 All rights reserved.
Shell ?
The interface between users/applications and the OS
Interactive interface to the system
Programming environment
You can use :
Graphical shells
Command line shells
© 2017 All rights reserved.
Linux shells
Command-line shells:
Sh, ash, bash, csh, dash, fish, ksh, zsh (and many, many more).
Graphic shells:
CDE, Gnome, KDE, Xfce, LXDE
© 2017 All rights reserved.
Bourne-again shell (bash)
Default POSIX shell for most Linux distros
Shell prompt is usually:
$ for normal users
# for root*
*user with „fear me, for I am root !” T-shirt
© 2017 All rights reserved.
Shell variables
Contain information important to the execution of the shell
Can be set:
Automatically on shell startup
Manually
$VARNAME
Examples:
$PS1
$SHELL
$PATH
…
© 2017 All rights reserved.
PATH
aka “search path”
Shell variable
Maintains a list of paths for usual commands
Aditional paths can be added
© 2017 All rights reserved.
Commands
Commands can be
internal (included in shell). E.g.: cd, pwd, echo
external (on disk)
Commands included in the shell are executed by default
Output may be different for external commands
Commands start applications, programs, scripts and utilities
Commands not in PATH can be executed with the fully qualified filename
Commands are typed in at shell prompt:
Example : [raducostin@localhost ~]$ ls -lsah
© 2017 All rights reserved.
Command structure
Linux is case sensitive!
Valid command components:
command (Ex. ls )
options (switches) (Ex. ls –lsa )
arguments (Ex. ls –lsa /home )
Switches can be chained
© 2017 All rights reserved.
Command structure -2-
Options (switches)
Short: -e -r –I
Can be chained: -erl
Long: --exclude-results, --recursive, --long-names
Options with parameters
-w 80
--width=80
arguments
file1 file2 file3
© 2017 All rights reserved.
Command history (1)
History is a list of executed commands
History size is set by HISTSIZE shell variable (default is 500)
Can be viewed with „history” command
© 2017 All rights reserved.
Command history designators
!! Spoken as bang-bang, this command refers to the most recent command. The
exclamation point is often called bang on Linux and Unix systems.
!n Refer to command n from the history. Use the history command to display these
numbers.
!-n Refer to the current command minus n from the history.
!string Refer to the most recent command starting with string.
!?string Refer to the most recent command containing string.
^string1^st Quick substitution . Repeat the last command, replacing the first occurrence of
ring2 string1 with string2.
© 2017 All rights reserved.
Command history (2)
Key bindings
Ctrl-p - Previous line (also up arrow)
Ctrl-n - Next line (also down arrow)
Ctrl-b - Back one character (also left arrow)
Ctrl-f - Forward one character (also right arrow)
Ctrl-a - Beginning of line
Ctrl-e - End of line
Ctrl-l - Clear the screen, leaving the current line at the top of the screen
Alt-< - Top of history
Alt-> - Bottom of history
Ctrl-d - Delete character from right
Ctrl-k - Delete (kill) text from cursor to end of line
Ctrl-y - Paste (yank) text previously cut (killed)
Alt-d - Delete (kill) word
Ctrl-r-text - Reverse search for text
Ctrl-s-text - Forward search for text
© 2017 All rights reserved.
More tricks
Command substitution
using either $(command) or `command`
Recursivity
commands with recursive option built-in (e.g. cp)
using find for recursivity
© 2017 All rights reserved.
Help and MAN pages
Ways to get additional help on command, usage and switches:
--help switch (available on some commands)
info command (available on some commands and distros)
man pages (always available, user’s best friend)
© 2017 All rights reserved.
Interpreting man syntax
ls [OPTION]... [FILE]...
[ARG] = optional argument (can be omitted)
ARG… = command accepts multiple arguments
Remember that options can be bundled together:
-l –r –a -lra
© 2017 All rights reserved.
Linux Administration
3. Working with Files.
Streams and Redirects
© 2017 All rights reserved.
PART I – BASIC FILE MANAGEMENT
© 2017 All rights reserved.
Basic file management
Filesystem objects
File and directory management commands
File-naming wildcards
© 2017 All rights reserved.
Filesystem objects
inode – identification information for a file system object
Most common objects in filesystem:
Files
Directories
Filesystem hierarchy standard (FHS)
/ vs. root
© 2017 All rights reserved.
Navigation
Directory tree referred to as a „path”
Absolute path vs. relative path
current directory [.], parent directory [..] and home directory [~]
pwd – Print working directory
cd – change directory
Ex. cd [directory]
ls – list files and directories
which - shows the full path of (shell) commands.
© 2017 All rights reserved.
Management commands
Objects are constantly created, read, modified, copied, moved and deleted:
touch – creates an empty file
mkdir – creates an empty directory
cp – copy one or more files to another location
mv – move or rename files and directories
rm – delete one or more files from filesystem
file – determine file type
© 2017 All rights reserved.
Links
links – linux version of shortcuts
soft links vs. hard links
ln – create links
ln file1 file2 – hard link
ln –s file1 file2 – soft link
© 2017 All rights reserved.
Wildcards
Wildcard Description
Match zero or more characters
*
Example: x* matches x, xy, xyz, x.txt, xy.txt, xyz.c,…
Match exactly one character.
?
Example: x? matches files or directories xx, xy, xz, but not x and not xyz.
Match any single character from string. Can also use a range.
[string]
Example: x[yz] matches xy and xz.
x[a-e] matches xa and xc, but not xf or xz.
Match any single character not in string.
[!string]
Example: x[!yz] matches xa and x1 but does not match xy or xz.
Create strings f1, f2, f3, etc. For example, file_{one,two,three} yields the strings file_one,
{f1,f2,f3...}
file_two, and file_three.
© 2017 All rights reserved.
PART II – STREAMS AND REDIRECTS
EVERYTHING IS A FILE
© 2017 All rights reserved.
Standard I/O – File descriptors
Standard I/O – shell capability to control and direct program input, output and error
File descriptors:
Standard Input (stdin/file descriptor 0) – text input stream. By default it is attached to
keyboard.
Standard Output (stdout/file descriptor 1) – text output stream. By default it is attached
to terminal.
Standard Error (stderr/file descriptor 2) – also a text output stream but used exclusively
for errors. By default it is attached to terminal.
© 2017 All rights reserved.
Pipe and redirects
Pipe ( | ) is used to tie output of one program to the input of another
Example: grep "01523" order* | less
Redirection can also occur to and from files.
You can redirect output of one program to a file:
ls –lsah > ls.txt
You can also redirect a file to the input of a program:
mail -s "inode list" jdean < in.txt
© 2017 All rights reserved.
Standard I/O redirection
Redirection function Syntax for bash
Send stdout to file. ( > or 1> ) cmd > file or cmd 1> file
Send stderr to file. ( 2> ) cmd 2> file
Send both stdout and stderr to file. cmd > file 2>&1
Send stdout to file1 and stderr to file2. cmd > file1 2> file2
Receive stdin from file. cmd < file
Append stdout to file. cmd >> file or cmd 1>> file
Append stderr to file. cmd 2>> file
Append both stdout and stderr to file. cmd >> file 2>&1
Pipe stdout from cmd1 to cmd2. cmd1 | cmd2
Pipe stdout and stderr from cmd1 to cmd2. cmd1 2>&1 | cmd2
© 2017 All rights reserved.
xargs, backtick and a cup of tee
xargs – used to receive stdin and pass it as command-line parameters to a specified
command.
Ex. find /home/user/ -name "*~" | xargs rm
backtick (`) not (‘) – shell replaces the content between backticks with the output
Ex. rm `find /home/user/ -name "*~"`
$(command) – same result, easier to read!
tee – read from standard input and write both to one or more files and to standard
output (analogous to a tee junction in a pipe).
© 2017 All rights reserved.
LPIC-1
Regular Expressions
© 2017 All rights reserved.
Remember – shell wildcards
Wildcard Description
Match zero or more characters
*
Example: x* matches x, xy, xyz, x.txt, xy.txt, xyz.c,…
Match exactly one character.
?
Example: x? matches files or directories xx, xy, xz, but not x and not xyz.
Match any single character from string. Can also use a range.
[string]
Example: x[yz] matches xy and xz.
x[a-e] matches xa and xc, but not xf or xz.
Match any single character not in string.
[!string]
Example: x[!yz] matches xa and x1 but does not match xy or xz.
Create strings f1, f2, f3, etc. For example, file_{one,two,three} yields the strings file_one, file_two, and
{f1,f2,f3...}
file_three.
© 2017 All rights reserved.
Wildcards vs. RegExps
Wildcards are NOT regular expressions!
Similar syntax, yet not identical
Wildcards (aka “file globs”):
expanded by the shell before command runs
used to refer to filenames
Regular expressions:
used by various tools (grep, sed, perl, etc.)
used to search and process text
© 2017 All rights reserved.
Patterns
Regular expressions are text patterns composed of literals and metacharacters
Literals (plain text)
a,b,c,@,&,…
Metacharacters (special characters)
.,+,*,[],…
© 2017 All rights reserved.
RegExps flavours
Various tools implement various flavours regular expressions
In the beginning there were the tools…
… then came the attempts at standardization
Same basic idea: matching text based on patterns
Small differences in syntax
© 2017 All rights reserved.
Basic metacharacters
() : grouping (see also backreferences)
. : any character (except a newline)
[] : range of characters
^ : in a range, negates the range
| : matches the regex before or after the vertical bar
+ : 1 or more instances of preceding regex
* : 0 or more instances of preceding regex
? : preceding regex can be present or not
© 2017 All rights reserved.
Examples
[bc]at
matches bat, cat
does not match hat, Cat
[a-z]ole
matches role, sole, pole
does not match Role
(rat|RAT)
matches rat or RAT
© 2017 All rights reserved.
Partial matches
Remember that partial matches are OK!
“at”
matches bat, cat, hat
does not match set
[bch]at
matches bat, cat, hat, that
does not match rat
© 2017 All rights reserved.
More examples
[^bc]at
matches hat, rat
does not match bat, cat
might match at (depending on previous character)!
[Cc]onfidential
matches confidential,Confidential
does not match confident
[Rr][Aa][Tt]
matches rat, Rat, RAT, raT
does not match hat, haT, Hat
© 2017 All rights reserved.
And even more examples
tr?oll
matches toll, troll, stroll
does not match stall, trill
tra+p
matches trap, traap, traaaaaaap,…
does not match trip, trp
sto*p
matches stop, stoop, stp
does not match step
© 2017 All rights reserved.
Character classes
[:alnum:] Alphanumeric [a-zA-Z0-9]
[:alpha:] Alphabetic [a-zA-Z]
[:blank:] Spaces or Tabs
[:cntrl:] Control characters
[:digit:] Numeric digits [0-9]
[:graph:] Any visible characters
[:lower:] Lowercase [a-z]
[:upper:] Uppercase [A-Z]
[:print:] Noncontrol characters
[:punct:] Punctuation characters
[:space:] Whitespace
[:xdigit:] Hex digits [0-9a-fA-F]
Valid within ranges! Ex: grep “[[:alpha:]01]” words
© 2017 All rights reserved.
Anchoring text
^ : beginning of text/line
$ : end of text/line
\< \> : start/end of word
the backslashes are required!
© 2017 All rights reserved.
Advanced matching
{m,n}: between m and n instances of preceding regex
\:
escapes metacharacters
gives special meaning to some literals
backreferences:
\1,\2… : match the group numbered 1, 2…
the order is determined by opening parentheses
© 2017 All rights reserved.
Examples
(abc)\1
matches abcabc
does not match abcabd, acabd
(.{2,3})\1
matches abab, abcabc, xyxy, xabxab
does not match abcdabcd, abba
ab\[cd\]\?
matches ab[cd]?
does not match abc, abcd, ab
© 2017 All rights reserved.
Regexps building blocks
Metacharacters and literals specify:
Position anchors
^, $
Character sets
[], rat, [:alpha:], .at
Quantity modifiers
?, *, {m,n}
© 2017 All rights reserved.
grep
Once upon a time there was ed…
and ed supported g/RE/p… (global / regular expression / print)
and this became grep
grep [options] regex [files]
if files are not specified, uses stdin (useful in pipes!)
© 2017 All rights reserved.
grep options
-c : display only a count of matched lines
-h : display lines, but do not include filenames for multiple-file input
-i : case-insensitive match
-n : display line numbers
-v : invert matching (print lines that do not match expression)
-E : use extended regular expression syntax
© 2017 All rights reserved.
Regexps vs. globbing revisited
# ls
abc abc1
abd
grep abc* *
is expanded to grep abc abc1 abc abc1 abd before execution!
Solution:
grep “abc*” * or
grep ‘abc*’ *
© 2017 All rights reserved.
grep examples
grep –i “linux” *
search for linux, Linux, LINUX, linuX… in all the files in the current directory
display filename and matching line if found
grep –c “rats?” rat*
search for rat or rats in all the files named rat, rats, ratlist… in the current directory
display a count of matching lines
gotcha: this will also match rating and aberration!
Solution?
anchoring the match!
© 2017 All rights reserved.
Linux Administration
3. Software Management
© 2017 All rights reserved.
PART I - INSTALLING FROM SOURCE
© 2017 All rights reserved.
Tarballs (not as scary as it sounds)
Source files – usually archived in a tarball.
usually a .tar.gz file (tar + gzip)
tar – used to store multiple files in a single archive
gzip/gunzip – used to compress/uncompress a file
bzip2 – smaller compressed files, at the cost of additional CPU overhead
© 2017 All rights reserved.
Compiling Open Source Software
configure - script that creates the correct Makefile for your system
Makefile – defines the correct parameters for compilation
make – builds targets from source files.
The ultimate target – the executable file(s)
make install - copies the resulting files to the appropriate place in the filesystem
© 2017 All rights reserved.
PART II – SHARED LIBRARIES
© 2017 All rights reserved.
Shared Libraries
Many of the functions required by programs are linked from system libraries:
disk functions
memory functions
various other functions
Statically linked programs contain the code from the libraries. Such a program stands alone,
requiring no additional code at runtime.
Dynamically linked programs load the necessary code from the libraries as needed.
Dynamically linked libraries are shared among many applications and are thus called shared
libraries
© 2017 All rights reserved.
Shared Library Dependencies
If the required shared libraries don't exist or can't be found,
the dynamically linked program will fail to run.
Installing the correct libraries should eliminate such problems.
ldd <programs>- Display shared libraries required by each of
the programs.
© 2017 All rights reserved.
Linking Shared Libraries
ld.so - looks for dependencies in the executable being loaded and attempts to satisfy
any unresolved links to system-shared libraries.
To add the new library entry to the ld cache, first add its directory to the ld.so.conf
file, which contains directories to be indexed by the ldconfig utility.
ldconfig - Update the ld.so cache file with shared libraries specified on the
command line, in /usr/lib and /lib, and in the directories found in /etc/ld.so.conf.
© 2017 All rights reserved.
PART III – DEBIAN PACKAGE MANAGEMENT
© 2017 All rights reserved.
Debian Package Management Overview
Each Debian package contains program and configuration files,
documentation, and noted dependencies on other packages.
The names of Debian packages have three common elements,
including:
Package name - short and descriptive.
When multiple words are used in the name, they are separated by hyphens.
Version number - The format varies from package to package, but most
are numeric (major.minor.patchlevel ).
File extension - .deb by default
© 2017 All rights reserved.
Managing Debian Packages – dpkg
The original Debian package management tool is dpkg,
dpkg operates directly on .deb package files and can be used to automate the
installation and maintenance of software packages.
Frequently used actions:
Action Description
-i package_file Install the package package_file
-l [pattern] List installed package files that match pattern
-L package List files installed from package.
--purge package Remove everything for package.
-r package Remove everything except configuration files for package
-s package Report the status of package
-S pattern Search for a filename matching pattern from installed packages
--unpack package Unpack package_file, but don't install the package it contains.
© 2017 All rights reserved.
Managing Debian Packages – apt-get
apt-get – part of Advanced Package Tool (APT) management system
Maintains a database of package information
Can automatically upgrade packages and their dependencies as new package
releases become available.
apt-get uses /etc/apt/sources.list to determine where packages should be obtained.
Frequently used commands:
Command Description
dist-upgrade Upgrade automatically to new versions of Debian Linux.
install Install or upgrade one or more packages by name.
remove Remove specified packages
update Fetch a list of currently available packages. This is typically done before any changes are
made to existing packages.
upgrade Upgrade a system's complete set of packages to current versions safely. This command is
conservative and will not process upgrades that could cause a conflict or break an
existing configuration; it also will not remove packages.
© 2017 All rights reserved.
Managing Debian Packages – dselect/alien
dselect - an interactive, menu-driven, frontend tool for dpkg
that lets you interactively manage packages
alien - Convert to or install a non-Debian (or "alien")
package. Supported package types include Red Hat .rpm,
Stampede .slp, Slackware .tgz, and generic .tar.gz files.
© 2017 All rights reserved.
PART IV – REDHAT PACKAGE MANAGEMENT
© 2017 All rights reserved.
RPM Overview
RPM automates the installation and maintenance of software
packages. Built into each package are program files, configuration
files, documentation, and dependencies on other packages.
RPM packages have four common elements:
Name - short and descriptive.
If multiple words are used, they are separated by hyphens.
Version - The format of package versions varies from package to package,
but most are numeric (major.minor.patchlevel ).
Revision - release number for the package.
Architecture
i386, i586, i686, noarch…
© 2017 All rights reserved.
rpm command (1)
rpm - The rpm command provides for the installation, removal,
upgrade, verification, and other management of RPM packages.
rpm modes :
rpm -i [options ] (also rpm --install)
rpm -U [options ] (also rpm --upgrade)
rpm -e [options ] (also rpm --uninstall)
rpm -q [options ] (also rpm --query)
rpm -V [options ] (also rpm --verify)
rpm –F [options] (also rpm –-freshen)
© 2017 All rights reserved.
rpm command (2)
Frequently
Option used
Description install and upgrade options:
--force Allows the replacement of existing packages and of files from previously installed packages;
for upgrades, it allows the replacement of a newer package with an older one
-h Prints a string of 50 hash marks (#) during installation as a progress indicator. (--hash)
--nodeps Allows you to install a package without checking for dependencies. (Not a good ideea!)
--test Runs through all the motions except for actually writing files
-v[v] Sets [really] verbose mode.
© 2017 All rights reserved.
rpm command – part II
uninstall mode is used to remove installed packages
from the system. By default, rpm uninstalls a package
only if no other packages are dependent on it.
Frequently used uninstall options:
Option Description
--nodeps Skip dependency checking (not a good idea!)
--test Verify that a package can be uninstalled correctly without breaking
other dependencies prior to making the attempt.
© 2017 All rights reserved.
rpm command – part III
Query mode - Installed packages and raw package files can be
queried using the rpm -q command.
Frequently used query options:
Option Description
-a Display a list of all packages installed on the system. (--all)
-f file Display the package that contains a particular file. (--file )
-p package Query a package file.
-c List only configuration files. (--configfiles)
-d List only documentation files. (--docfiles)
-i package Not to be confused with the install mode. Display information about an installed
package, or when combined with -p, about a package file..
-l package List all of the files contained in package. (--list)
-R List packages on which this package depends. (--requires)
© 2017 All rights reserved.
rpm command – part IV
Verify mode - Files from installed packages can be compared against their
expected configuration from the RPM database by using rpm -V.
Frequently used verify options:
Option Description
--nofiles Ignores missing files.
--nomd5 Ignores MD5 checksum errors.
--nopgp Ignores PGP checking errors.
© 2017 All rights reserved.
Package management summary
Debian RedHat
Package installation dpkg rpm
apt
Package management / updater yum
(apt-get)
GUI Tools synaptic pirut
GUI Updater update-manager pup
© 2017 All rights reserved.
Linux Administration
4. Disks and Partitions.
© 2017 All rights reserved.
Physical Disks
Entries in the /dev directory
Actual communication is handled using a major/minor device number combination
/proc/devices – list of major numbers
/dev/hdXY – IDE HDDs
/dev/sdXY – SCSI (or SATA!) HDDs
X = a,b,c… - disk number
Y = 1,2,3… - partition number
Examples:
/dev/hdb1
Primary slave IDE disk, first partition
/dev/sdc5
Third SCSI/SATA disk, first logical partition
© 2017 All rights reserved.
Partitions (MBR)
Each HDD can have:
Up to 4 primary partitions
One of them can be an extended partition
With multiple logical partitions on it
512B Primary Primary Primary Ext
EBR Logical Logical
© 2017 All rights reserved.
MBR Limitations
Only 4 primary partitions (or 3 + 1 extended)
MBR supports only 1 byte partition type codes, which are not standardized
collisions
MBR stores partition sector information using 32-bit LBA values.
Along with the (commonly used) 512 byte sector size, this limits the maximum
addressable size of the disk to 2 TiB.
© 2017 All rights reserved.
GPT
Uses GUIDs to identify partition types (no collisions).
Arbitrary number of partitions - depends on space allocated for the GPT. By
default, 128 partitions.
Uses 64bit LBA for storing Sector numbers (2 ZiB max disk size).
Stores a backup header and partition table at the end of the disk.
CRC32 checksums to detect errors and corruption of the header and
partition table.
© 2017 All rights reserved.
PSA: Binary Prefixes!
Decimal Binary
Value SI Value IEC JEDEC
1000 k kilo 1024 Ki kibi K kilo
2 2
1000 M mega 1024 Mi mebi M mega
3 3
1000 G giga 1024 Gi gibi G giga
4 4
1000 T tera 1024 Ti tebi – –
5 5
1000 P peta 1024 Pi pebi – –
6 6
1000 E exa 1024 Ei exbi – –
7 7
1000 Z zetta 1024 Zi zebi – –
8 8
1000 Y yotta 1024 Yi yobi – –
© 2017 All rights reserved.
Partitions (GPT)
Location Purpose
First 512B Protective MBR - Same as a normal MBR but the 64-byte area
contains a single 0xEE type Primary partition
Next 512B Primary GPT Header
Next 16KiB Primary GPT Table
Last 512B Secondary GPT Header
16KiB before Secondary GPT Table
© 2017 All rights reserved.
View existing partitions
fdisk –l
Device Boot Start End Blocks Id System
/dev/sda1 * 1 1013 8136891 8e Linux LVM
/dev/sda2 1014 1044 249007+ 5 Extended
/dev/sda5 1014 1044 248976 83 Linux
© 2017 All rights reserved.
Speaking of fdisk…
The main tool used for partitioning in Linux
There are many other tools for the same purpose (parted, QtParted, diskdruid,
Yast…)
CLI-only or GUI
Dynamic partition resize
© 2017 All rights reserved.
Using fdisk
Start up fdisk by specifying the physical disk to work on (e.g. /dev/sda):
# fdisk /dev/sda
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help):
© 2017 All rights reserved.
fdisk Commands
p – Print the current partitions
You can also use fdisk –l device, to skip interactive mode
n – create a New partition
d – Delete a partition
t – change partition Type
default: 0x83 – Linux
use for swap partitions, or partitions destined for another OS (Windows FAT32, NTFS, etc.)
l – List partition types
a – mark partition as Active
? or m – help
q – Quit without saving changes
w – Write changes and quit
© 2017 All rights reserved.
GPT fdisk
GPT fdisk is a set of text-mode utilities for editing GPT disks.
It consists of gdisk, sgdisk and cgdisk which are equivalent fdisk (used for MBR disks)
© 2017 All rights reserved.
Using gdisk
Same as fdisk
# fdisk /dev/sda
GPT fdisk (gdisk) version 0.8.8
Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: not present
Command (? for help):
© 2017 All rights reserved.
gdisk Commands
b back up GPT data to a file
c change a partition's name
d delete a partition
i show detailed information on a partition
l list known partition types
n add a new partition
o create a new empty GUID partition table (GPT)
p print the partition table
q quit without saving changes
s sort partitions
t change a partition's type code
v verify disk
w write table to disk and exit
? print this menu
© 2017 All rights reserved.
Filesystems
After creation, a partition needs to be formatted using a filesystem
Most common filesystems used in Linux:
Ext2fs (Second Extended File System)
Mostly replaced by ext3
Still used for flash media (no journal less writes)
Ext3fs (Third Extended File System)
Journaled file system
Eliminates the need for disk check after unclean shutdown
© 2017 All rights reserved.
Journaling
Many file system operations are not atomic
Example - deleting a file requires:
1. Removing the directory entry
2. Marking the space as free
Only step 1 storage leak (orphaned inode)
Only step 2 file is still visible, yet its contents can be overwritten
Solution - journaling:
record changes before applying them
replay changes if a crash occurs
© 2017 All rights reserved.
Filesystems (2)
Ext4fs (Fourth Extended File System)
Successor to ext3
Journaling filesystem
January 15, 2010 – Google announced it will upgrade its storage infrastructure from ext2 to ext4
Android 2.3 will use ext4 (instead of YAFFS)
ReiserFS
Optimized for large numbers of small files
JFS (Journaled File System)
developed by IBM for AIX and later for OS/2
OS/2 version donated to Linux
© 2017 All rights reserved.
Filesystems (3)
XFS (Extens File System)
created by SGI (Silicon Graphics) for IRIX OS
released under GPL to Linux in 2000
64-bit FS
263 – 1 bytes max file size
Btrfs
Development began at Oracle
GPL Licensed
inspired by ZFS
© 2017 All rights reserved.
Filesystems (4)
Other filesystems supported by Linux:
FAT16/FAT32
Used by DOS/Windows
NTFS
Preferred filesystem on Windows NT/200x/XP/Vista.
Support for read/write operations is provided by the NTFS-3G driver.
ISO-9660, Joliet, UDF
Used for optical media (CD-ROM, DVD-ROM, etc.)
© 2017 All rights reserved.
Creating a filesystem
mkfs.fstype <device>
or by using the frontend:
mkfs –t <fstype> <device>
For ext2 and ext3 – frontend:
mke2fs [-j] <device>
For MS-DOS filesystems:
mkdosfs (frontend for mkfs.vfat, mkfs.msdos)
© 2017 All rights reserved.
Swap partitions
Swap partition – used by Linux for implementing virtual memory mechanisms
Special partition type
Marked as swap space in /etc/fstab
Creating the swap space:
mkswap <device>
© 2017 All rights reserved.
Checking filesystems
fsck – frontend for fsck.ext2, fsck.ext3, etc
Syntax:
# fsck [-sACVRTNP] [-t fstype] [--] [options] <fs>
Common options:
-A : check all filesystems in /etc/fstab
-C : show progress indicator
-V :Verbose output
-t fstype : specify a particular filesystem type
Options specific to filesystems:
-f (force verify)
Final parameter – list of filesystems to be checked
© 2017 All rights reserved.
Files and filenames
Filenames in Linux
Can contain letters (uppercase and lowercase), numbers, and other characters
Should not contain characters with special meanings: * ? \
Are case-sensitive!!
Special filenames:
. , .. , ~
Wildcards:
? – one character
* - zero or more characters
[] – match a range of characters
© 2017 All rights reserved.
Wildcard examples
b??k
Matches book, back, bark
Does not match bk, brink
b*k
Matches bk, back, book, brink
Does not match blocks,nobook
b[a-e]ck
Matches back, beck, bdck
Does not match bnck, bask
© 2017 All rights reserved.
ls
ls (list)
Display all files in a given directory
Frequently used options:
-a (--all) : display all files (including hidden files)
--color : color files depending on their type
-l : display additional information (including permissions)
-R : recursively display the contents of subdirectories
© 2017 All rights reserved.
05.RAID. LVM
© 2017 All rights reserved.
RAID
© 2017 All rights reserved.
Hardware RAID vs. Software RAID
Hardware RAID
Not flexible
Fast
RAID subsystem is independent from host system.
Better performance than Software RAID
Expensive
Built-in RAID controller in hardware
Software RAID
Flexible, cheap
Easy to implement RAID
Implemented in kernel disk (block device) code
Performance dependent on host system (CPU and Memory)
Consumes around 25% of host system processing cycles
But, fast CPU help improve the performance of Software RAID.
© 2017 All rights reserved.
Performance Comparison
© 2017 All rights reserved.
Linux Software RAID
Supported in 2.6 Linux
kernel series by default Application
File System
RAID Levels
Linear mode Buffer Cache
RAID-0
RAID-1 Software RAID
RAID-4
Device Driver
RAID-5
Etc.
Hacking the source code
/usr/src/linux/drivers/md/raid0.c (or raid1.c, raid5 …)
© 2017 All rights reserved.
Example – RAID1
Configuration File File System /
Buffer Cache
/etc/raidtab
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
/dev/md
nr-spare-disks 0
persistent-superblock 1
device /dev/sdb6
raid-disk 0 /dev/sdb6 /dev/sdc5
device /dev/sdc5
raid-disk 1
Initialization
mkraid /dev/md0
© 2017 All rights reserved.
The md driver
Provides virtual devices
Created from one or more independent underlying devices
The basic mechanism to support RAIDs
Redundant arrays of inexpensive disks
© 2017 All rights reserved.
COMMON RAID LEVELS
RAID0
Striping
RAID5 (> 3 disks)
RAID1 Striped array with distributed parity
Mirroring
RAID6 (> 4 disks)
RAID4 (> 3 disks) Striped array with dual redundancy
Striped array with a parity device information
© 2017 All rights reserved.
COMMON RAID LEVELS
RAID1+0
Striped array of mirrored disks
RAID5+1
RAID0+1
Mirroring two RAID5s
Mirroring two RAID0s
RAID5+0
Striped array of RAID5s
© 2017 All rights reserved.
md pseudo RAID configurations
Linear (catenates multiple disks into a single one)
Multipath
A set of different interfaces to the same device (e.g., multiple disk controllers)
Faulty
A layer over a single device into which errors can be injected
© 2017 All rights reserved.
RAID Creation
> mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/hd[ac]1
Create /dev/md0 as RAID1
Consisting of /dev/hda1 and /dev/hdc1
© 2017 All rights reserved.
RAID Status
To check the status for RAIDs
See /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda5[0] sdb5[1]
979840 blocks [2/2] [UU]
md1 : active raid1 sda6[2] sdb6[1]
159661888 blocks [2/1] [_U]
[===>.................] recovery = 17.9%
(28697920/159661888) finish=56.4min speed=38656K/sec
unused devices: <none>
© 2017 All rights reserved.
md Super Block
Each device in a RAID may have a superblock with various information
Level
UUID
128 bit identifier that identifies an array
© 2017 All rights reserved.
Synchrony
An update may involve both the data block and the parity block
Implications
A RAID may be shut down in an inconsistency state
Resynchronization may be required at startup, in the background
Reduced performance
© 2017 All rights reserved.
Recovery
If the md driver detects a write error, it immediately disables that device
Continues operation on the remaining devices
Starts recreating the content if there is a spare drive
© 2017 All rights reserved.
Recovery
If the md driver detects a read error
Overwrites the bad block
Read the block again
If fails, treat it as a write error
Recovery is a background process
Can be configured via
/proc/sys/dev/raid/speed_limit_min
/proc/sys/dev/raid/speed_limit_max
© 2017 All rights reserved.
LVM
© 2017 All rights reserved.
LVM
LVM - Logical Volume Manager
First developed by HP for the HP-UX
Operating System
LVM is the default volume management
system in Red Hat Enterprise Linux
© 2017 All rights reserved.
Logical Volumes
Volume management creates a layer of abstraction over
physical storage providing much greater flexibility.
Hardware storage configuration is hidden from the software
© 2017 All rights reserved.
LVM Overview
Uses a collection of disks
A single volume can span multiple physical disks
Not all disks need to be the same size
Each disk is referred to as a „Physical Volume”
Physical Volumes are collected into Volume Groups.
A Volume Group is split into Logical Volumes
Logical Volumes contain the file system.
© 2017 All rights reserved.
LVM Terms
Physical Volume (PV) - a device (HDD) with some administrative data added to it
Physical Extent (PE) - a chunk of storage space located onto a physical volume
(PV)
Volume Group (VG) - A VG is a grouping of PVs into a common storage pool
Logical Volume (LV) - The resulting equivalent of a disk partition in a non-LVM
system
© 2017 All rights reserved.
LVM Terms - Example
© 2017 All rights reserved.
LVM Tools - PVs
pvcreate - initialize a block device to be used as a PV
lvmdiskscan - scans for block devices that may be used as PVs.
pvdisplay/pvs/pvscan - used to display properties of PVs
pvchange – change properties of PVs
pvremove – remove PVs
© 2017 All rights reserved.
LVM Tools – VG
vgcreate - create a volume group from one or more PVs
vgextend - adds additional PVs to an existing VG
vgdisplay/vgs - used to display properties of LVM volume groups
vgscan - scans all supported disk devices in the system looking for
PVs and VGs
vgreduce – removes PVs from VG
vgchange – used to change attributes for a VG
© 2017 All rights reserved.
LVM Tools – VG - cont.
vgremove – remove a VG
vgsplit - splits a VG and creates new VG
vgmerge – used to combine two VGs into a single VG
vgcfgbackup/vgcfgrestore – backup/restore metadata
vgrename – renames an existing VG
vgexport – can move an entire VG to another system.
vgmknodes – recreates a VG directory and LV special files.
© 2017 All rights reserved.
LVM Tools - LV
lvcreate – used to create a LV
lvconvert – convert a LV from linear to mirror or snapshot
lvs – report information about LV
lvdisplay – display attributes of a LV
lvreduce – reduce the size of a LV
lvchange – change attributes of a LV
lvrename – rename a LV
lvremove – remove a LV
lvextend – extend a LV
© 2017 All rights reserved.
GUI LVM Management
system-config-lvm
© 2017 All rights reserved.
Linux Administration
06. Linux Boot Process
© 2017 All rights reserved.
BOOTLOADERS
© 2017 All rights reserved.
OS Loading
MBR /boot
POST – BIOS – Look up Load bootloader Load rest of
Power on Hardware Test boot order Kernel
in memory bootloader
(stage1) (stage2)
© 2017 All rights reserved.
Bootstrapping
Starting the system
Process of loading kernel into memory
Boot Modes
Normal
Single User/Safe Mode
Rescue (on DVD)
© 2017 All rights reserved.
Bootloader
Software read and executed by the BIOS at system startup
Its purpose – loading the OS kernel in memory and running it
Located in the MBR (Master Boot Record) – first 512B of a partitioned HDD
The MBR contains the bootloader and the partition table
© 2017 All rights reserved.
How a Bootloader Works
Two possibilities:
1. The bootloader:
1. Scans the partition table and locates the partition marked as active
(bootable)
2. Loads the boot sector on that partition and executes it
3. The boot sector contains a secondary bootloader that continues
the process of locating, loading and running the kernel
2. The bootloader locates the OS kernel and executes it directly
(bypassing the secondary bootloader)
© 2017 All rights reserved.
How a Bootloader Works
© 2017 All rights reserved.
RH/CENTOS 6 BOOT PROCESS
/etc/rc.d/
BIOS /etc/inittab
rc3.d
/etc/rc.d/
Linux rc.sysinit RL Specific
GRUB init
Kernel /etc/inittab
/etc/rc.d/rc
Login /etc/rc.d/
Shell rc5.d
© 2017 All rights reserved. Source: http://nmc.nchu.edu.tw/linux/Linux_boot.htm
SYSTEMD DISTROS BOOT PROCESS
/etc/rc.d/
BIOS systemd rc3.d
Systemd
Linux
GRUB init RL Specific
Kernel /etc/
init/*.conf
Login /etc/rc.d/
Shell rc5.d
© 2017 All rights reserved. Source: http://nmc.nchu.edu.tw/linux/Linux_boot.htm
Boot Process
Power On Self Tests (POST)
Run Boot Loader(s)
Load & initialize kernel
Detect & configure devices
Fork system processes
(Stop if Single User mode)
Run startup scripts
Start multiuser operations
© 2017 All rights reserved.
LILO and GRUB
LILO – LInux LOader
GRUB – Grand Unified Bootloader
Both can be installed either in MBR or the boot sector of a partition
Installation in a partition requires an active partition or activation by a third-party
bootloader
Both can boot Linux or redirect the boot process to the partition corresponding to
another OS (including Windows)
LILO options are hardcoded in the boot sector, while GRUB uses external
configuration files
© 2017 All rights reserved.
Configuring GRUB Legacy
/etc/grub.conf
Some distributions may use /boot/menu.lst or /boot/grub/grub.conf
Disks are specified as (hdX,Y)
X=number of the disk (starting with 0)
Y=partition number (starting with 0)
© 2017 All rights reserved.
grub.conf
# grub.conf/menu.lst
➢ Global options:
• Default image to boot
#
# Global Options: • Timeout for default
# selection
• Background for GRUB
default=0 menu (splashimage=)
timeout=15
splashimage=/grub/bootima
ge.xpm.gz
© 2017 All rights reserved.
grub.conf (2)
# Kernel Image Options:
# Image options:
title Fedora (2.6.9)
root (hd0,0)
• Title (image name)
kernel /vmlinuz-2.6.9 ro • Root partition
root=/dev/hda5 mem=512M
initrd /initrd-2.6.9 • Kernel image location
title Debian (2.6.11)
• Chainloading for non-linux
root (hd0,0) OS
kernel (hd0,0)/bzImage-
2.6.11 ro root=/dev/hda6
# Other operating systems
#
title DOS
rootnoverify (hd0,1)
chainloader +1
© 2017 All rights reserved.
Configuring grub2
/boot/grub/grub.cfg
Important changes compared to GRUB Legacy include the following
The title keyword is replaced by menuentry.
The menu title is enclosed in quotation marks.
An opening curly brace ({) follows the menu title, and each entry ends with a closing curly brace
(}).
The set keyword precedes the root keyword, and an equal sign (=) separates root from the
partition specification.
The rootnoverify keyword has been eliminated; you use root instead.
Partitions are numbered starting from 1 rather than from 0. A similar change in disk
numbering is not implemented.
© 2017 All rights reserved.
Creating grub.cfg
Compiled using grub2-mkconfig
/etc/default/grub
Variables to change default settings
/etc/grub.d/*
00_header
10_linux
40_custom
© 2017 All rights reserved.
Installing grub
# grub-install <device>
<device> can be specified
As a filename: /dev/hda
As a GRUB disk/partition name: (hd0), (hd0,0)
The boot menu allow editing an option (by pressing “E”)
This can be used for password recovery
You may need to run update-grub after updating GRUB 2’s /etc-based configuration files.
© 2017 All rights reserved.
RUNLEVELS
© 2017 All rights reserved.
Runlevels
Runlevels define what services or processes should be running on the system
The init process can run the system in one of seven runlevels. The system runs only one runlevel at a
time.
Runlevel Description
0 Halt (shutdown)
1, s, S Single-User mode
2 Multiuser. On Debian this is the default runlevel. On Red Hat it is multiuser without networking.
3 Multi-User mode, console logins only. (not used in Debian).
4 Not used.
5 Multi-User mode, with display manager as well as console logins (X11) (not used in Debian)
6 Reboot
© 2017 All rights reserved.
System Processes
BSD Systems
swapper – PID 0
init – PID 1
pagedaemon – PID 2
Unix System V
sched – PID 0 (invisible under RedHat/CentOS)
init – PID 1
/etc/inittab
© 2017 All rights reserved.
/etc/inittab
The "/etc/inittab" file has information on which runlevel to start the system at and lists the
processes to be run at each runlevel.
Each runlevel can be configured by the system administrator.
Each runlevel has its own directory structure where you can define the order in which the services
start.
These directories are located in the /etc directory, under which you have rc1.d, rc2.d, rc3.d…. rc6.d
(coresponding to each runlevel)
Inside each directory are symbolic links that point to master initscripts found in /etc/init.d or
/etc/rc.d/init.d.
The actions of init for each runlevel are derived from Unix System V-style initialization
© 2017 All rights reserved.
Startup Services
Hostname
Timezone
Check the hard drives
Mount the hard drives
Remove files from /tmp
Configure network interfaces
Start daemons and network services
© 2017 All rights reserved.
init vs upstart – „old way”
init is level-driven – you allocate each service to a runlevel, and services are started up in blocks
based on which runlevel you boot into.
Within the runlevels, the names of the start and stop links govern the timing of the script.
Each /etc/rcx.d directory has a collection of softlinks to the start/stop service scripts in /etc/init.d/.
These look like this:
K20service -> /etc/init.d/service
S35service -> /etc/init.d/service
The numbers control the order, to avoid dependency problems: lower numbers are run first.
init cannot handle hardware that's plugged in after bootup.
init cannot handle networked filesystems which may not be available on boot
init cannot handle daemons which we'd like only to run when the hardware is available
© 2017 All rights reserved.
init vs. upstart – „new way”
upstart is event-based
Services can be started or stopped in response to other events occurring on the
system
Events are used to trigger tasks or services, collectively known as jobs.
upstart can also handle restarting services if they die unexpectedly (which init can't
do)
The upstart initialization program replaces /sbin/init. Upstart jobs are defined in the
/etc/init directory and its subdirectories.
On systems such as recent Fedora releases, /etc/inittab is likely to contain only the
id entry for the initdefault action. Recent Ubuntu systems do not have /etc/inittab
by default
Upstart also has the initctl command to allow interaction with the upstart init
daemon.
© 2017 All rights reserved.
systemd
Systemd is a system and service manager for Linux operating systems.
It is designed to be backwards compatible with SysV init scripts
Provides a number of features such as:
parallel startup of system services at boot time
on-demand activation of daemons
support for system state snapshots
In RH 7/Ubuntu 15.10+, systemd replaces Upstart as the default init system.
© 2017 All rights reserved.
Systemd - Units
Uses a dependency system between “units”
Requires/Wants
Conflicts
Before
After
Encapsulate objects relevant to booting and
maintenance
Configured in config files
May be tied through symbolic links
© 2017 All rights reserved.
Systemd – Unit Types
Different unit types control different aspects of the
operating system
service: handles daemons
socket: handles network sockets
target: Logical grouping of units (example: runlevel)
device: expose kernel devices
mount: controls mount points of the files system
automount: mounts the file system
snapshot: references other units (similar to targets)
© 2017 All rights reserved.
Systemd Unit File Section
[Unit]
Description
Requires
Wants
Conflicts
Before
After
© 2017 All rights reserved.
Systemd Service Section
[Service]
Type=
simple|oneshot|forking|dbus|notify|idle
ExecStart
ExecReload
ExecStop
Restart=
no|on-success|on-failure|on-abort|always
© 2017 All rights reserved.
Systemd Install Section
[Install]
Wantedby=
Used to determine when to start
(e.g. Runlevel)
© 2017 All rights reserved.
Managing Runlevel Services
The SysV startup scripts in the runlevel directories are symbolic links back to the
original script.
You can also modify which programs are active in a runlevel by editing the link
filenames.
Numerous utility programs are available to help you manage these links, such
as chkconfig, ntsysv, update-rc.d, and rc-update.
In systemd you can use systemctl command.
© 2017 All rights reserved.
Commands (upstart vs. systemd)
Operation Upstart Systemd
Start service start $job systemctl start $unit
Stop service stop $job systemctl stop $unit
Restart service restart $job systemctl restart $unit
Status of services initctl list systemctl status
Check config init-checkconf /tmp/foo.conf systemd-analyze verify <unit_file>
Show job env initctl list-env systemctl show-environment
tail log tail –f /var/log/upstart/$job.log journalctl -u $unit -f
© 2017 All rights reserved.
Using chkconfig
Option Explanation
--list list the services and their applicable runlevels
--list <name> view runlevels for a particular service
--level <levels> <name> alter runlevels for a particular service (on, off or reset to
on|off|reset default value)
--add <name> register a service and add appropriate start and stop links
in the runlevel directories.
© 2017 All rights reserved.
Using update-rc
update-rc.d [options] name action
Action Effect
remove Removes links in runlevel-specific directories to the named service.
defaults Creates links to start the service in runlevels 2, 3, 4, and 5, and to stop it in
runlevels 0, 1, and 6.
start NN runlevels Creates a link to start the service in the specified runlevels, using the
sequence number NN.
stop NN runlevels Creates links to stop the service in the specified runlevels, using the
sequence number NN.
enable [runlevel] Modifies existing runlevel links to enable the service in the specified
runlevel. If no runlevel is specified, runlevels 2, 3, 4, and 5 are modified.
disable [runlevel] Modifies existing runlevel links to disable the service in the specified
runlevel. If no runlevel is specified, runlevels 2, 3, 4, and 5 are modified.
© 2017 All rights reserved.
systemctl
Command Effect
systemctl start|stop|restart Used to start|stop|restart a service (not reboot persistent)
<service>
systemctl condrestart
Restarts if the service is already running.
<service>
systemctl status <service> Tells whether a service is currently running.
systemctl (or) systemctl list- Used to list the services that can be started or stopped
unit-files --type=<service>
systemctl enable|disable Turn the service on|off, for start at next boot, or other trigger.
<service>
systemctl is-enabled Used to check whether a service is configured to start or not in the current
<service> environment.
© 2017 All rights reserved.
using the systemd journal
Binary format is (rightfully) controversial.
Run “addgroup $USER systemd-journal” for access.
Log-reading tools are simple:
journalctl -xn
journalctl -p err
journalctl /usr/sbin/cron
systemctl status
systemctl is-failed <service>
systemctl --failed
© 2017 All rights reserved.
Upstart-native
Scripts in /etc/init directory (in /etc/event.d on early upstart versions)
Ex.
start on (filesystem
and started hal
and tty-device-added KERNEL=tty7
and (graphics-device-added or stopped udevtrigger))
stop on runlevel [016]
Before changing your runlevel you should type initctl reload to have Upstart reread its
configuration files.
© 2017 All rights reserved.
Systemd – Runlevels
Runlevel is defined through a symbolic to one of the runlevel targets
Runlevel Target
Runlevel 3:
/lib/systemd/system/multi-user.target
Runlevel 5:
/lib/systemd/system/graphical.target
Change Runlevel:
Remove current link /etc/systemd/system/default.target
Add a new link to the desired runlevel
© 2017 All rights reserved.
/etc/init/*.conf directives
exec
script
start on <event>
stop on <event>
task
respawn
See man 5 init for more
© 2017 All rights reserved.
“Events”
control-alt-delete
power-status-changed
startup
runlevel <runlevel>
started <job>
stopped <job>
© 2017 All rights reserved.
Runlevel commands
When a Linux system starts, the default runlevel is determined from
the id: entry in /etc/inittab
To determine what runlevel your system is in use the „runlevel” command
init sends signals to the executing init process, instructing it to change to a
specified runlevel
init n - The number of the runlevel, n, can be changed to an integer from 1 through 6
You can also use telinit (a link to init).
shutdown command brings the system down in a secure, organized fashion
shutdown takes the system to single-user mode by default
-h is used to halt the system, -r to reboot.
ex.: shutdown -r +5 System maintenance is required
© 2017 All rights reserved.
Changing runlevels
To make a permanent change, you can edit /etc/inittab and change the default
level (sysvinit ) or use /etc/systemd/system/default.target (systemd)
If you need to bring system up in a different runlevel for one boot you can:
Edit the kernel line in GRUB
Add a parameter after the selected system name in LILO
If you need to switch between runlevels while system is running use „init” command.
© 2017 All rights reserved.
Linux Administration
07. Users and groups. Process management.
© 2017 All rights reserved.
PART 1 – USERS & GROUPS
© 2017 All rights reserved.
11/7/2017 179
User accounts
Linux is a multiuser operating system.
It is necessary to differentiate between the users so that their private files
can be kept private.
An account is all the files, resources, and information belonging to one
user.
The basic user database in a Unix system is the text file, /etc/passwd,
which lists all valid usernames and their associated information.
"The similarities of sysadmins and drug dealers: both
measure stuff in Ks, and both have users."
© 2017 All rights reserved.
passwd file – part I
Each line in the file contains information for a single system account :
Username - The first field on a line is a unique username for the person or service
using the account.
Password - Each username has an associated password. For security reasons, most
systems store user passwords in a separate /etc/shadow file.
UserID - Each username requires a unique user identifier, or UID. The UID is simply a
nonnegative integer.
© 2017 All rights reserved.
passwd file – part II
GroupID - Each username has a default group identifier, or GID. The GID is also a nonnegative integer.
Full name (or other comment) - The user's full name or other information is stored as plain text. This
field may contain spaces.
Home directory - The home directory is the default directory in the filesystem for the user's account.
Default shell - This field specifies the default shell for the user or service, which is the shell that runs when
the user logs in or opens a shell window.
The root account has UID and GID 0, which gives it global privilege on the system.
© 2017 All rights reserved.
Groups and the Group File
Linux groups are a mechanism to manage a collection of computer system users.
Groups are similar to users in their administration and are defined in the file
/etc/group
Groups can be assigned to logically tie users together for a common security,
privilege and access purpose.
© 2017 All rights reserved.
/etc/group
Like the passwd file, the group file contains colon-separated fields
Group name - Each group must have a unique name.
Group password - Groups can have passwords for their membership.
Group ID - Each group requires a unique GID. Like a UID, a GID is a
nonnegative integer.
Group member list - The last field is a list of group members by
username, separated by commas.
© 2017 All rights reserved.
shadow & gshadow
/etc/passwd and /etc/group are readable by everyone on the system.
/etc/shadow and /etc/gshadow are readable only by root.
A group password can be used to allow access to a group by a user
that is not a member of the group.
Account users can use the newgrp command to change their default
group and enter the group password.
© 2017 All rights reserved.
User and Group Management Commands - 1
Option Description
useradd Define
-C comment [options] user field,
the comment - Create
usually thethe account
user'suser on the
name.
system. Both
-d homedir
system defaults and specified options define how
Use homedir as the user's home directory.
-m
the account is configured.
Create and populate the home directory.
-S shell Use shell as the default for the account.
-D List (and optionally change) system default values
© 2017 All rights reserved.
User and Group Management Commands - 2
usermod [options] user - Modify an existing user account. The usermod
command accepts many of the same options useradd does.
-L - Lock the password, disabling the account.
-U - Unlock the user's password, enabling the user to log in to the system.
userdel [-r] user - Delete an existing user account. When combined with
the -r option, the user's home directory is deleted.
groupadd group - Add group to the system.
groupmod [option] group - Modify the parameters of group.
groupdel group - Delete group from the system.
passwd [options] username - Interactively set the password for
username.
© 2017 All rights reserved.
Passwords
gpasswd groupname - Interactively set the group password for
groupname.
pwconv - Convert a standard password file to a password and shadow
password combination.
pwunconv - Revert from a shadow password configuration to a standard
password file.
grpconv - Convert a standard group file to a group and shadow group
combination
grpunconv - Revert from a shadow group configuration to a standard group
file.
© 2017 All rights reserved.
File properties
Every file has:
An owner (user)
A group
Viewing the owner/group:
# ls –l
-rwxr-xr-x 1 root root 1076 2010-01-28 00:42 process.pl
drwxr-xr-x 4 bogd bogd 4096 2009-11-02 15:23 school
Changing the owner:
# chown [options] [newowner][:newgroup] filenames
-r : change the owner for an entire subtree recursively
Changing the group:
# chgrp [options] newgroup filename
© 2017 All rights reserved.
File permissions
Every file has associated with it a 10-character string denoting permissions:
# ls –l
-rwxr-xr-x 1 root root 1076 2010-01-28 00:42 process.pl
drwxr-xr-x 4 bogd bogd 4096 2009-11-02 15:23 school
First character denotes the file type:
- : normal file
d : directory
l : symbolic link
p : named pipe
s : socket
b : block device
c : character device
© 2017 All rights reserved.
File permissions (2)
# ls –l
-rwxr-xr-x 1 root root 1076 2010-01-28 00:42 process.pl
drwxr-xr-x 4 bogd bogd 4096 2009-11-02 15:23 school
The other 9 characters are 3 blocks of 3 characters, denoting permissions for owner / group / other users
The permissions are:
r : read
w : write
x : execute
Each permission can be present or absent
The result – a representation on 9 bits, that can be written as 3 octal digits:
rwxr-xr-x = 755
rw-r--r-- = 644
© 2017 All rights reserved.
File permissions (3)
eXecute permission (x) for directories = the right to change to the directory
Write permission (w) for directories = the right to create, delete or rename files in
the directory
Even if the user does not have permissions on the files!
Special permissions
SUID (Set User ID – rwsr-xr-x) – executes a file using the permissions of the file’s owner
SGID (Set Group ID – rwxr-sr-x) – executes a file using the permissions of the file’s
owner group
© 2017 All rights reserved.
Changing file permissions
# chmod [options] [mode[,mode…]] filename…
The new permissions can be specified:
numerically (3 octal digits, plus an extra initial digit for special permissions: SUID-4, SGID-
2, sticky-1):
# chmod 675[0] program
by specifying the entity and rights that are added (+), withdrawn (-), or applied exactly (=):
# chmod a+x program
# chmod u+rwx,g-rx,o=x program
© 2017 All rights reserved.
Default permissions
Defined by umask (user mask)
octal value that is subtracted from 777 (for directories) / 666 (for files)
Viewing current umask:
# umask
changing the umask:
# umask u=rwx,g=rx,o=rx (specify the permissions for the newly created files!)
# umask 022 (specify the numeric umask)
root can change the umask for various users by using /etc/profile
© 2017 All rights reserved.
Changing file attributes
Supplemental attributes (depending on filesystem):
a : append only
c : compressed
i : immutable (cannot be renamed or deleted)
j : data journaling
s : secure deletion
chattr can add (+), remove (-) or specify exactly the attributes for a file:
# chattr +i important.txt
# chattr =aj data.txt
© 2017 All rights reserved.
Tips and tricks – ACLs
ACLs are an addition to the standard Unix file permissions (r,w,x,-)
for User, Group, and Other.
ACLs give users and administrators flexibility and fine-grained control
over who can read, write, and execute files.
To enable ACLs, add „acl” option in /etc/fstab for a filesystem and
remount that filesystem.
getfacl – lists ACLs for an object
setfacl – sets ACLs for an object
© 2017 All rights reserved.
PART 1I – PROCESS MANAGEMENT
© 2017 All rights reserved.
11/7/2017 197
Processes
Every program is a process (even the shell).
First process started – init
the ultimate parent of all other processes.
Every shell command starts one or more processes (referred
to as child processes).
© 2017 All rights reserved.
Process attributes
Lifetime - the length of time the process takes to execute.
When a process terminates, it is said to die (which is why the program used to manually
signal a process to stop execution is called kill)
Process ID (PID) - unique number assigned to every process
User ID (UID) and Group ID (GID) - associated with the user who started the
process, determine the rights the process has
© 2017 All rights reserved.
Process attributes (2)
Parent process ID (parent PID) - PID of the process that created the process in
question.
If the parent dies, the child is “adopted” by init (PID 1)
Environment - list of variables and their associated values.
Variables are inherited from parents.
Current working directory - the process will read and write files in this directory
unless they are explicitly specified to be elsewhere
© 2017 All rights reserved.
Process Monitoring - ps
ps - generates a one-time snapshot of the current processes on standard output.
options:
Option Description
-a Show processes that are owned by other users and attached to a terminal. Normally, only the current user's
processes are shown.
-f "Forest" mode, which is used to display process family trees.
-l Long format, which includes priority, parent PID, and other information.
-u User format, which includes usernames and the start time of processes.
-w Wide output format, used to eliminate the default output line truncation. Useful for the -f option.
-x Include processes without controlling terminals. Often needed to see daemon processes and others
not started from a terminal session.
-C cmd Display instances of command name cmd.
-U user Display processes owned by username user.
© 2017 All rights reserved.
Process Monitoring - pstree
pstree - displays a hierarchical list of processes in a tree format (similar with ps –f )
options:
Options Description
-a Display command-line arguments used to launch processes.
-c Disable the compaction of identical subtrees.
-G Use the VT100 line-drawing characters instead of plain characters to display the tree. This yields a much
more pleasing display but may not be appropriate for printing or paging programs.
-h Highlight the ancestry of the current process (usually the shell). The terminal must support highlighting for
this option to be meaningful.
-n The default sort order for processes with the same parent is alphanumerically by name. This option
changes this behavior to a numeric sort by PID.
-p Include PIDs in the output.
© 2017 All rights reserved.
Process Monitoring – top I
top - offers output similar to ps, but in a continuously updated display.
Dashes are optional in top options:
Option Description
-b Run in batch mode. This is useful for sending output from top to other programs or to a file. It
executes the number of iterations specified with the -n option and terminate.
-d delay Specify the delay in seconds between screen updates. The default is five seconds.
-i Ignore idle processes, listing only the "interesting" ones taking system resources.
-n num Display num iterations and then exit, instead of running indefinitely.
-q Run with no delay. If the user is the superuser, run with highest possible priority. This option causes top
to update continuously and will probably consume any idle time your CPU had.
-s Run in secure mode.
© 2017 All rights reserved.
Process Monitoring – top II
some interactive options:
Option Description
Ctrl-L Screen refresh
h Generate a help screen.
k Kill a process.You will be prompted for the PID of the process and the signal to send it.
n Change the number of processes to show.You will be prompted to enter an integer number.
q Quit the program.
r Change the priority of a process (renice).
s Change the delay in seconds between updates.
© 2017 All rights reserved.
Signaling Active Processes
signals – numeric integer predefined messages sent to the process either by the
kernel or by a user through interprocess communication
When a process receives a signal (usually sent with „kill” command), it can (or may
be forced) to take action.
There are more than 32 signals defined for normal process use in Linux. Each signal
has a name and a number (the number is sent to the process, the name is only for
convenience)
© 2017 All rights reserved.
Linux Signals
Name Number Description
HUP 1 Hang up. It is used by many daemons to cause the configuration file to be reread.
INT 2 Interrupt; stop running. This signal is sent when you type Ctrl-C.
KILL 9 Kill; stop unconditionally and immediately. Sending this signal is a drastic measure, as it
cannot be ignored by the process. This is the "emergency kill" signal.
TERM 15 Terminate, nicely if possible. This signal is used to ask a process to exit gracefully.
TSTP 20 Stop executing, ready to continue. This signal is sent when you type Ctrl-Z.
CONT 18 Continue execution. This signal is sent to start a process stopped by SIGTSTP or
SIGSTOP. (The shell sends this signal when you use the fg or bg commands after
stopping a process with Ctrl-Z.)
© 2017 All rights reserved.
Shell Job Control
Job control - the ability of the shell (with support of the kernel)
to stop and restart executing commands, as well as place them
in the background
A foreground program – one that is attached to the terminal.
© 2017 All rights reserved.
Job Control (2)
When executing in the background, you have no input to the process other than sending it signals.
Background process = job.
Numbered sequentially, starting with 1
Place programs in the background:
& at the end of the command
Ctrl-Z , bg
Bring a job to the foreground:
fg
List jobs:
jobs
© 2017 All rights reserved.
Job control commands
bg [jobspec] - Place jobspec in the background
fg [jobspec] - This command places the specified job in the
foreground.
jobs [jobspec] - List the active jobs. The optional jobspecs
argument restricts output to information about those jobs.
© 2017 All rights reserved.
Modify Process Execution Priorities
Process priority
the PRI column in top or ps –l
The administrator has the ability to prioritize process execution.
Each process's priority level is constantly and dynamically raised and lowered by the kernel according
to a number of parameters, such as how much system time it has already consumed and its status.
nice number – specified by user, and used to adjust process priority
Default – 0
Higher number = lower priority
© 2017 All rights reserved.
nice / renice
nice [-number] [command] - alter nice number at process
start
number for normal users: 1 to 19
number for superuser: -20 to 19
default number: 10
renice [+|-]nicenumber [option] targets - alter nice number
of running processes
targets – numeric PIDs
Remember: You can also renice processes interactively using top's
text interface (r command)
© 2017 All rights reserved.
PART III – SCHEDULING JOBS
© 2017 All rights reserved.
11/7/2017 212
SYSTEM LOGGING
SYSLOG, RSYSLOG, SYSLOG-NG
© 2017 All rights reserved.
System logs
Text files
Stored (usually) in the /var/log/ directory
Can also be sent to a remote server
Offer various (configurable) levels of information on the working of:
The operating system
Other applications
© 2017 All rights reserved.
The syslogd daemon
Usually part of the sysklogd package
Together with the klogd daemon, which handles kernel logging
Unified logging system
Receives messages from various sources (applications, remote devices, etc) and
(usually) stores them into files
The syslog protocol – standardized in RFC 5424 (obsoletes RFC 3164)
© 2017 All rights reserved.
Logging facilities
Identify the origin of the message
Frequently used facilities:
Name Facility
kern Kernel
user Regular user processes
mail Mail system
lpr Line printer system
auth Authorization system
daemon Other system daemons
news News subsystem
uucp UUCP subsystem
local0... local7 Reserved for site-specific use
mark Timestamp - sends out a message every 20 minutes
© 2017 All rights reserved.
Logging priorities
Identifies the severity of the event
Valid priorities:
Number Name Description
0 Emergency System is unusable
1 Alert Immediate action required
2 Critical Critical condition
3 Error Error condition
4 Warning Warning condition
5 Notice Normal but significant condition
6 Informational Informational message
7 Debug Debug-level message
© 2017 All rights reserved.
/etc/syslog.conf
Identifies:
Which messages should be logged
Where the messages are sent:
files (and other file-like devices, e.g. terminals)
message to user(s)
pipe to a program
another host (over the network)
© 2017 All rights reserved.
/etc/syslog.conf -2-
Each line consists of:
a message selector
facility . priority
an action
daemon.* -/var/log/daemon.log
mail.info -/var/log/mail.info
*.=debug -/var/log/debug
*.emerg *
*.* @remote.logging.host
mail.*; *.info |/dev/xconsole
© 2017 All rights reserved.
Logfile examples
General format:
date/time hostname program: message
root@thermite:~# tail /var/log/auth.log
Mar 6 21:20:49 thermite sshd[10514]: Failed password for invalid user bureau
from 222.87.204.11 port 41506 ssh2
Mar 6 21:20:53 thermite sshd[10518]: Invalid user jasmin from 222.87.204.11
Mar 6 21:48:19 thermite sshd[11375]: Invalid user ant from 121.14.31.2
Mar 6 21:48:24 thermite sshd[11379]: Invalid user office from 121.14.31.2
© 2017 All rights reserved.
Manually logging data
logger [-isd] [-f file] [-p pri] [-t tag] [-u socket] [message ...]
Options:
-f file – log contents of file
-s – also log to stderr
-t – specify custom tag
-p pri – specify priority (and facility)
© 2017 All rights reserved.
Trust no one…
bogd@thermite:~$ logger -p auth.crit -t "ATTACK"
Remote attack succeeded!
bogd@thermite:~$ tail -n 1 /var/log/auth.log
Mar 9 20:38:04 thermite ATTACK: Remote attack
succeeded!
Moral of the story:
Don’t trust anyone 100%
Not your logs
And definitely not your users! ☺
© 2017 All rights reserved.
SCHEDULING JOBS
CRON
© 2017 All rights reserved.
cron
Job scheduler
Runs jobs at specified date/time(s)
System cron jobs (e.g. logrotate)
Run as root
System-wide maintenance tasks
User cron jobs
Run as a particular user
© 2017 All rights reserved.
Scheduling a cron job
System job:
/etc/crontab
files in /etc/cron.d/
/etc/cron.INTERVAL/ - hourly, daily, weekly…
Must specify the “run as” user
User job:
crontab –e
Final crontabs are stored in /var/spool/cron/crontabs/ (or similar)
© 2017 All rights reserved.
crontab file format
Environment setting lines
SHELL=/bin/sh
cron commands
Min Hr DoM Mth DoW [USER] COMMAND
* = any
*/X = run at intervals of X
Output from the commands (if any) is sent to the owner of the crontab by mail
© 2017 All rights reserved.
Sample system crontab
Default crontab on Ubuntu 10.10 (ignoring anacron tests for now)
root@vm-ubuntu:/var/log# cat /etc/crontab
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
17 * * * * root cd / && run-parts –report /etc/cron.hourly
25 6 * * * root cd / && run-parts –report /etc/cron.daily
47 6 * * 7 root cd / && run-parts –report /etc/cron.weekly
52 6 1 * * root cd / && run-parts –report /etc/cron.monthly
© 2017 All rights reserved.
Sample user crontab
bogd@thermite:~$ crontab -l
# m h dom mon dow command
*/5 * * * * /home/bogd/update_app/update.py
© 2017 All rights reserved.
More info on crontab
man 1 crontab
The crontab program
man 5 crontab
The crontab file format
man cron
© 2017 All rights reserved.
at
Runs command(s) at a specified time in the future
Relies on the atd demon
atq – print pending jobs
atrm – remove job from queue
© 2017 All rights reserved.
at example
root@vm-ubuntu:~# at "now+1 minute"
at> echo "done" > 1min.txt
at> <EOT>
job 3 at Thu May 10 13:26:00 2011
root@vm-ubuntu:~# atq
3 Thu May 10 13:26:00 2011 a root
root@vm-ubuntu:~# at "13:27"
at> echo "done" > fixed_time.txt
at> <EOT>
job 4 at Thu May 10 13:27:00 2011
© 2017 All rights reserved.
at example -2-
root@vm-ubuntu:~# date
Thu Mar 10 13:26:57 EET 2011
root@vm-ubuntu:~# cat fixed_time.txt
cat: fixed_time.txt: No such file or
directory
root@vm-ubuntu:~# date
Thu Mar 10 13:27:00 EET 2011
root@vm-ubuntu:~# cat fixed_time.txt
done
© 2017 All rights reserved.
Linux Administration
8. Basic Networking
© 2017 All rights reserved.
NETWORKING FUNDAMENTALS
TCP/IP, TCP, UDP, ICMP, PORTS
© 2017 All rights reserved.
OSI – TCP/IP
TCP/IP OSI OSI
ISO created standard
Application
Application
OSI Reference Model was intended to
Presentation
be prescriptive, hence being a
reference model Session
Transport Transport
TCP/IP
Internet Network
DARPA created model
Network Data link
Access adopted as the internet model
Physical
TCP/IP model and related protocols
are maintained by the IETF
© 2017 All rights reserved.
TCP/IP
TCP/IP
Data Encoding
Application
Provides services to the users
Transport Provides a transport service for application data.
Internet Attempts to ensure that messages reach their
destination system using the most efficient route.
Network
Access Is used to move packets between interfaces of two
different hosts on the same link
© 2017 All rights reserved.
Data Transmission
Application Data
Presentation
Session
PDU Data Data Data Data
Data Transmission
Transport
PDU Segment T Header Data
Network
PDU N Header T Header Data
Packet
Data link
PDU Frame DL Header N Header T Header Data Trailer
Physical
PDU Bits 010101001010010010010010
© 2017 All rights reserved.
Packet segmentation
packet segmentation is the process of dividing a data
packet into smaller units for transmission over the network
Destination host rearranges segments according to
sequence number
Any cumulative stream not acknowledged is retransmitted
© 2017 All rights reserved.
Addressing
Physical Data Link Network Transport Session
MAC Port
Sync Bits IP address numbers
Address
© 2017 All rights reserved.
Addressing (Data Link)
© 2017 All rights reserved.
Addressing (Network)
© 2017 All rights reserved.
Addressing (Transport)
© 2017 All rights reserved.
Addressing (Session)
© 2017 All rights reserved.
Transport layer protocols
▪ The most well-known transport protocols are
(Transmission Control Protocol) and UDP (User Datagram
Protocol)
TCP: Reliable, connection-oriented
Transport
UDP: Unreliable, connectionless
© 2017 All rights reserved.
Port numbers (I)
To differentiate the segments and datagrams for each application, TCP and UDP
have header fields (port numbers) that can uniquely identify these applications.
The source port number is associated with the originating application on the local
host.
The destination port number is associated with the destination application on the
remote host.
A socket pair, consisting of the source and destination IP addresses and port
numbers, is unique and identifies the conversation between the two hosts.
© 2017 All rights reserved.
Port numbers (II)
A port number is a 16 bits number.
The Internet Assigned Numbers Authority (IANA) assigns port
numbers. IANA is a standards body that is responsible for assigning
various addressing standards.
There are different types of port numbers:
Număr port Tip port
From 0 to 1023 Well-known ports
From 1024 to 49151 Registered ports
From 49152 to 65535 Private and dynamic ports
© 2017 All rights reserved.
Port numbers (III)
© 2017 All rights reserved.
TCP
TCP is a connection-oriented protocol, described in RFC 793
TCP incurs additional overhead to gain functions. Additional functions
specified by TCP are the same order delivery, reliable delivery, and
flow control.
Each TCP segment has 20 bytes of overhead in the header
encapsulating the Application layer data, whereas each UDP segment
only has 8 bytes of overhead.
© 2017 All rights reserved.
Three Way Handshake(I)
In TCP connections, the host
serving as a client initiates the 1 2
session to the server.
Send SYN Receive SYN
Establishes that the destination Seq = x Seq = x
device is present on the Send SYN
network Seq = y,
Ack = x + 1
Verifies that the destination Receive SYN
Seq = y,
device has an active service Ack = x + 1
Informs the destination device Send ACK
that the source client intends to Ack = y + 1
Receive SYN
establish a communication Ack = y + 1
session on that port number
© 2017 All rights reserved.
TCP – Segmentation
When services send data using TCP, segments may arrive at their destination
out of order.
Sequence numbers are assigned in the header of each packet and the
segments are reassembled into the original order.
During session setup, an initial sequence number (ISN) is set.
The receiving TCP process places the data from a segment into a receiving
buffer.
Segments are placed in the proper sequence number order and passed to
the Application layer when reassembled.
© 2017 All rights reserved.
UDP
UDP is a simple protocol that provides the basic Transport layer
functions.
This does not mean that applications that use UDP are always
unreliable. It simply means that these functions are not provided by
the Transport layer protocol and must be implemented elsewhere if
required.
Some applications, such as online games or VoIP, can tolerate some
loss of some data.
UDP does not keep track of sequence numbers the way TCP does.
UDP has no way to reorder the datagrams into their transmission
order. See the figure.
© 2017 All rights reserved.
IP Addressing
Historically, RFC1700 grouped the unicast ranges into 5 classes
Class First byte First byte Network networks hosts
(binary) prefix
A 1-127 00000000- /8 2^7 2^24-2
01111111 128 16,677,214
B 128-191 10000000- /16 2^14 2^16-2
10111111 16,384 65,534
C 192-223 11000000- /24 2^21 2^8-2
11011111 2,097,150 254
D 224-239 11100000- N/A N/A N/A
11101111
E 240-255 11110000- N/A N/A N/A
11111111
© 2017 All rights reserved.
Not so great expectations...
1970 2011
• 32 bits ~ 4 billions IP • Almost 2 billions users.
addresses • Less than 5% ip addresses
• ARPANET available
• “32 bits should be
enough address space”
© 2017 All rights reserved.
Solutions: Classless
Classful allocation of address space often wasted many addresses, which exhausted
the availability of IPv4 addresses.
Even though this classful system was all but abandoned in the late 1990s, you will see
remnants of it in networks today.
With the classless system, address blocks appropriate to the number of hosts are
assigned to companies or organizations without regard to the unicast class.
© 2017 All rights reserved.
Solutions: Private ip addresses
blocks of addresses that are Prefix Range
used in networks that require
/8 10.0.0.0-
limited or no Internet access.
10.255.255.255
Packets using these addresses as /12 172.16.0.0-
the source or destination 172.31.255
should not appear on the public /16 192.168.0.0-
Internet. 192.168.255.255
© 2017 All rights reserved.
NAT
With services to translate private addresses to public addresses, hosts
on a privately addressed network can have access to resources across
the Internet.
NAT allows the hosts in the network to "borrow" a public address for
communicating to outside networks. While there are some limitations
and performance issues with NAT, clients for most applications can
access services over the Internet without noticeable problems.
192.168.1.2:32001 86.1.1.1:32001
private
public
192.168.1.3:32001 86.1.1.1:32002
192.168.1.4:32001 86.1.1.1:32003
© 2017 All rights reserved.
Default Route
The default route is used as a "catch all" route when a more
specific route is not available.
Default destination is 0.0.0.0/0
© 2017 All rights reserved.
Loopback
127.0.0.1/32 (but the entire 127.0.0.0/8 space is reserved).
The loopback is a special address that hosts use to direct traffic to themselves.
The loopback address creates a shortcut method for TCP/IP applications and
services that run on the same device to communicate with one another.
You can also ping the loopback address to test the configuration of TCP/IP on the
local host.
© 2017 All rights reserved.
Link local
IPv4 addresses in the address block 169.254.0.0 to 169.254.255.255
(169.254.0.0 /16) are designated as link-local addresses.
These addresses can be automatically assigned to the local host by
the operating system in environments where no IP configuration is
available.
A host must not send a packet with an IPv4 link-local destination
address to any router for forwarding and should set the IPv4 TTL for
these packets to 1.
© 2017 All rights reserved.
IANA
Internet Assigned Numbers Authority (IANA) (http://www.iana.net) is the
master holder of the IP addresses.
Until the mid-1990s, all IPv4 address space was managed directly by the
IANA.
At that time, the remaining IPv4 address space was allocated to various
other registries to manage for particular purposes or for regional areas:
AFRNIC - Africa
APNIC - Asia, Pacific
LACNIC - South America & Caraibe
ARIN- North America
RIPE NCC –Central and Eastern Europe, Middle East
© 2017 All rights reserved.
NETWORKING CONFIGURATION
INTERFACES, CONFIG FILES
© 2017 All rights reserved.
Interfaces
A computer must contain at least one network interface to be
considered part of a network.
ifconfig is used to create and configure network interfaces and their
parameters.
Without parameters, a list of all active interfaces and their
configurations is displayed.
Parameters:
address - The interface's IP address.
netmask - The interface's subnet mask.
up - Activate an interface (implied if address is specified).
down - Shut down the interface.
© 2017 All rights reserved.
Network config files
/etc/hosts - This file contains simple mappings between IP addresses
and names and is used for name resolution.
/etc/nsswitch.conf - This file controls the sources used by various
system library lookup functions, such as name resolution.
/etc/host.conf - This file controls name resolution sources for pre-
glibc2 systems.
/etc/resolv.conf - This file controls the client-side portions of the
DNS system.
/etc/networks - Like /etc/hosts, this file sets up equivalence
between networks and names.
© 2017 All rights reserved.
Configuration commands
host [options] name [server]
Look up the system with IP address or name on the DNS server
hostname [localname]
domainname [nisname]
dnsdomainname
Set or display the current host, domain, or node name of the system.
netstat [options]
Depending on options, netstat displays network connections, routing tables, interface statistics,
masquerade connections, netlink messages, and multicast memberships.
© 2017 All rights reserved.
All-In-One Networking Tool
ip - An alternative to the ifconfig, route, and various other commands for many
purposes.
ip [ OPTIONS ] OBJECT { COMMAND | help }
ex. ip route list
Object Description
link Performs actions on network hardware; similar to some ifconfig functions
addr Associates or disassociates a device with an address; similar to ifconfig
addrlabel Displays or adjusts addresses in an IPv6 network
route Displays or adjusts the routing table; similar to some route functions
rule Displays or adjusts firewall table rules; similar to some iptables functions
neigh Displays or adjusts ARP entries
monitor Monitors network for activity
© 2017 All rights reserved.
Routes - 1
route is used to establish static routes to specific networks or hosts (such as the
default gateway) after an interface is configured.
Option Description
-n Numeric mode; don't resolve hostnames.
-C Display the kernel routing cache.
-F Display the kernel routing table (the default behavior without add or delete ).
-host | -net Specify that target is a single host or a net. Mutually exclusive.
gw IP packets for target are routed through the gateway, which must be reachable.
netmask Specify the mask of the route to be added.
© 2017 All rights reserved.
Routes -2
When used to display routes, the following routing table columns are printed:
Column Description
Destination The destination network or host.
Gateway The gateway address. If no gateway is set for the route, an asterisk (*) is displayed by default.
Genmask The netmask for the destination. 255.255.255.255 is used for a host and 0.0.0.0 is used for the
default route.
Route status flags !, D, G, H, R, M, U (reject, dynamic, use gw, target is host, modified, reinstated, up).
Metric The distance in hops to the target.
Ref Number of references to this route.
Use A count of lookups for the route.
Iface The interface to which packets for this route are sent.
© 2017 All rights reserved.
NETWORK TROUBLESHOOTING
PING, TRACEROUTE, ROUTE, NETSTAT
© 2017 All rights reserved.
ICMP
Current version is ICMPv4
Ping uses the ICMP protocol (Layer 3) to check for connectivity.
The ping command provides a method for checking the protocol stack and IPv4
address configuration on a host.
The ping command will not always pinpoint the nature of the problem, but it can help
to identify the source of the problem.
© 2017 All rights reserved.
ICMP Messages
Echo
Confirm connectivity using a pair of messages: Echo-Request , Echo-
Reply
Time Exceeded
Sent to the originator of a packet in order to inform it that the
packet’s TTL has reached 0 before reaching the destination
Redirect
Sent to notify the originator of traffic about a better route to a
destination
© 2017 All rights reserved.
netstat
--interface or -i parameter – Interface Information (similar to ifconfig)
--route or -r – Routing Information (similar to route)
--masquerade or -M – Masquerade Information about connections mediated by
Linux’s NAT features
--program or -p – Program Use attempts to provide information about the
programs that are using network connections.
--all or -a – All Connections - It causesnetstat to display information about the ports
that server programs open to listen for network connections.
© 2017 All rights reserved.
Linux Administration
9. Writing Basic Shell Scripts
© 2017 All rights reserved.
“If it’s worth doing more than once, it’s
worth automating”
© 2017 All rights reserved.
Why shell scripting?
Combine lengthy and repetitive sequences of commands into a single,
simple command
Generalize a sequence of operations
Create new commands using combinations of utilities
Create customized datasets on the fly, and call applications (e.g. matlab, sas,
idl, gnuplot) to work on them
© 2017 All rights reserved.
Shell scripts – typical uses
System boot scripts (/etc/init.d)
System administration
Computer maintenance
User account creation
etc, etc, etc
Application package installation tools
Application startup scripts
Especially unattended applications (e.g. started from cron or at)
Automating a CLI process
© 2017 All rights reserved.
Creating a shell script
A shell script is just another text file
created using a text editor: pico, nano, vi, joe, mcedit…
The kernel knows how to run a script file by looking at the first line:
#!/bin/bash
also known as “shebang”, “hashbang”, “hashpling”…
notice there are no spaces between the characters!
This line is not used by the shell (# marks a comment)
© 2017 All rights reserved.
Running the script
Mark the file as executable
bogd@thermite:~/curs$ cat script.sh
#!/bin/bash
echo "Hello world"
bogd@thermite:~/curs$ chmod a+x script.sh
Run it like any other executable file
bogd@thermite:~/curs$ ./script.sh
Hello world
© 2017 All rights reserved.
Shell script building blocks
External commands
Variables
Conditional expressions (tests)
Control structures (if, for, while)
© 2017 All rights reserved.
Running external commands
Just like running them from the shell command line
It is recommended to provide the full path to the executable
The user running the script might not have the same $PATH as you!
Remember:
options
-e -r –I -erl
--exclude-results, --recursive, --long-names
options with parameters
-w 80
--width=80
arguments
file1 file2 file3
© 2017 All rights reserved.
Frequently used commands
File manipulation commands
cp
mv
rm
Text processing commands
grep
find
cut
sed
tr
echo
© 2017 All rights reserved.
Shell variables
Variables can be set:
from command-line arguments to the script
$1, $2…
from the environment
$PATH, $SHELL…
internally by the script
myvar=valoare
echo $myvar
read myvar (from keyboard input)
All variables are strings, and unset variables default to the empty string
© 2017 All rights reserved.
Variables example
bogd@thermite:~/curs$ cat vars.sh
#!/bin/bash
myvar="hello"
echo "My variable is <$myvar>"
echo "Current path is <$PATH>"
echo "Argument 0 is <$0>"
echo "Argument 1 is <$1>"
echo "Argument 2 is <$2>"
© 2017 All rights reserved.
Variables example -2-
bogd@thermite:~/curs$ ./vars.sh 1 2 3
My variable is <hello>
Current path is </usr/local/sbin:/usr/local/bin>
Argument 0 is <./vars.sh>
Argument 1 is <1>
Argument 2 is <2>
bogd@thermite:~/curs$ ~/curs/vars.sh 1 2 3
My variable is <hello>
Current path is </usr/local/sbin:/usr/local/bin>
Argument 0 is </home/bogd/curs/vars.sh>
Argument 1 is <1>
Argument 2 is <2>
© 2017 All rights reserved.
Conditional expressions (tests)
Placed in square brackets ( [ … ] )
“test EXPR” is equivalent to [ EXPR ]
A few of the (many!) possible tests:
String tests
[ string1 = string2 ] – the strings are equal
[ -z string ] – string has zero length
Integer tests
[ int1 –gt int2 ] – int1 is greater than int2
File tests
[ -e file ] – file exists
[ -d file ] – file exists and is a directory
Spaces around the brackets are REQUIRED!
“man test” or “info coreutils ‘test invocation’” for a complete list of tests
© 2017 All rights reserved.
Control structures - if
Syntax:
if EXPR
then
COMMANDS
fi
“then” and “fi” are separate commands, and must be on
separate lines (or separated by “;”)
© 2017 All rights reserved.
Example – if and test
bogd@thermite:~/curs$ cat test.sh
#!/bin/bash
if [ -e $0 ] #Check my own filename
then
echo "I do indeed exist!"
else
echo "I am not there…"
fi
bogd@thermite:~/curs$ ./test.sh
I do indeed exist!
© 2017 All rights reserved.
Loops - for
Syntax:
for VAR in LIST
do
COMMANDS
done
“do” and “done” are separate commands, and must be on
separate lines (or separated by “;”)
© 2017 All rights reserved.
Example - for
bogd@thermite:~/curs$ cat for1.sh
#!/bin/bash
for file in file1 file2 for1.sh
do
if [ -e $file ]
then
echo "File $file exists!"
else
echo "File $file does not exist!"
fi
done
bogd@thermite:~/curs$ ./for1.sh
File file1 does not exist!
File file2 does not exist!
File for1.sh exists!
© 2017 All rights reserved.
Numbered loops
Old method: seq
bogd@thermite:~/curs$ cat for2.sh
#!/bin/bash
for i in $(seq 1 3)
do
echo $i
done
Deprecated as of bash 3.x!!
© 2017 All rights reserved.
Numbered loops -2-
New method: {a..b}
bogd@thermite:~/curs$ cat for3.sh
#!/bin/bash
for i in {1..3}
do
echo $i
done
© 2017 All rights reserved.
Numbered loops -3-
New method: three-expression (C-style)
bogd@thermite:~/curs$ cat for4.sh
#!/bin/bash
for ((i=1;i<=3;i++))
do
echo $i
done
© 2017 All rights reserved.
While loops
Syntax:
while EXPR
do
COMMANDS
done
“do” and “done” are separate commands, and must be on
separate lines (or separated by “;”)
© 2017 All rights reserved.
Functions
Defining a function:
function_name() {
COMMANDS
}
Calling a function:
function_name
© 2017 All rights reserved.
Example - functions
bogd@thermite:~/curs$ cat safecopy.sh
#!/bin/bash
check_file(){
if [ -e $1 ]
then
echo "Target file $1 exists!"
exit
fi
}
copy_file(){
cp $1 $2
}
check_file $2 # Check if destination already exists!
copy_file $1 $2
© 2017 All rights reserved.
Arithmetic operations
Used for counters, or simply calculating operation results
expr - all-purpose evaluator
Requires escaping of parantheses and some operators ( + , * , etc)
let - simple version of expr
double parantheses: $((…))
© 2017 All rights reserved.
Arithmetic operations - examples
bogd@thermite:~$ echo $y
0
bogd@thermite:~$ let y++
bogd@thermite:~$ echo $y
1
bogd@thermite:~$ let y=y+10
bogd@thermite:~$ echo $y
11
bogd@thermite:~$ y=$(expr $y + 1)
bogd@thermite:~$ echo $y
12
bogd@thermite:~$ y=`expr $y + 1`
bogd@thermite:~$ echo $y
13
bogd@thermite:~$ y=$(($y+10))
bogd@thermite:~$ echo $y
23
© 2017 All rights reserved.
Floating-point operations
bogd@thermite:~$ echo 3.17 + 2.15 | bc
5.32
© 2017 All rights reserved.
Linux Administration
10. Security Administration
© 2017 All rights reserved.
HOST SECURITY
PASSWORDS, ROOT ACCESS, LIMITS
© 2017 All rights reserved.
Securing passwords
Requires user (and management!) involvement
Password recommendations:
use strong passwords
change passwords frequently
use shadow passwords
keep passwords secret
use secure remote login
use separate passwords on separate systems
© 2017 All rights reserved.
Password strength
Poor passwords:
names (family/friends/pets), date of birth, telephone numbers, favourite shows, etc.
ANY word that is found in a dictionary!
Strong passwords:
upper/lowercase, punctuation, digits
at least 8 characters!
Suggestion:
“The quick brown fox jumps over the lazy dog!” Tqbfj0tld!
“Trust none of the people around you!” Tn0tp4y!
© 2017 All rights reserved.
Password changes
Can be enforced by the system
Balance between security and usability:
too rare password changes less security
too frequent password changes users will pick less secure passwords (or write them
down)
© 2017 All rights reserved.
Shadow passwords
Password hashes have been moved from the world-readable /etc/passwd to the
more secure /etc/shadow.
Shadow passwords also add support for password aging and account expiration
For more details, see man chage .
© 2017 All rights reserved.
Keeping passwords secret
Shoulder surfing
© 2017 All rights reserved.
Social Engineering
© 2017 All rights reserved.
Root login
Worst option:
Direct login as root:
not recommended
no trace in logs as to who actually typed the root password
A little better:
su - , or su –c COMMAND – leaves a trace in the logs
Much better:
sudo COMMAND – runs a single command as superuser
requires the user’s password
© 2017 All rights reserved.
Setting limits
Done via PAM module (pam_limits)
Can set limits for:
logged-in users
CPU
memory
/etc/security/limits.conf
© 2017 All rights reserved.
limits.conf
domain type item value
Domain:
username
@groupname
* (everyone)
Type:
hard (cannot be exceeded)
soft (can be modified by users)
© 2017 All rights reserved.
limits.conf
domain type item value
Item:
core – size of core files
data – size of a program’s data area
cpu – CPU time of a process (minutes)
maxlogins – number of simultaneous logins
priority – default process priority
…
Value:
actual value of the limit
© 2017 All rights reserved.
limits.conf example
@limited hard cpu 2
Applies to users in the “limited” group
Hard limit
Limits CPU time used by a process
After 2 minutes of CPU time, the process will be killed
© 2017 All rights reserved.
ulimit
ulimit [options [limit]]
Some possible options:
-c : core file size
-f : file limits
-u : process limits
-H/-S : hard/soft limit
-a – show current limits
Bash built-in command, only affects bash and programs launched from it!
© 2017 All rights reserved.
SUID/SGID files
Files run with the permissions of the owning user (SUID) / owning group (SGID)
Security risk (especially if SUID root!)
Searching for SUID/SGID files:
find / –perm +6000 -f
© 2017 All rights reserved.
CONFIGURING SSH
© 2017 All rights reserved.
Remote access tools
“Traditional” remote access tools:
telnet
VNC
X
Unencrypted traffic
Vulnerable to sniffing
© 2017 All rights reserved.
SSH
Offers a fully-encrypted remote connection, plus:
File transfer
Tunnelling
Drawbacks:
extra CPU consumption
Most common server - OpenSSH
© 2017 All rights reserved.
SSH Configuration
/etc/ssh/sshd_config
Options:
Protocol – version of the SSH protocol
PermitRootLogin – whether to accept or not direct root logins
…
man sshd_config
© 2017 All rights reserved.
SSH Keys
Public/private keys
Public server keys are stored on the client at first connection, and user is warned on
server key change
stored in ~/.ssh/known_hosts
Keys can be generated using ssh-keygen
© 2017 All rights reserved.
Controlling SSH Access
TCP Wrappers
Allow you to use /etc/hosts.allow and /etc/hosts.deny to control access
Requires SSH to be compiled with TCP Wrappers support (or run from a superserver)
Firewalls
Just an example:
iptables -A ssh_filter -m limit --limit 2/minute --limit-burst 3 -j ACCEPT
/etc/nologin
If present, only root can login
© 2017 All rights reserved.
Copying files
SCP (SecureCopy)
Syntax:
scp [options] local_file
[user@]remote_host:/path/to/destination.file
© 2017 All rights reserved.
Key-based authentication
Allows you to use a public/private key combination in order to login to another machine
1. Generate the key pair:
ssh-keygen –t rsa –b 2048
HIGHLY recommended to protect the key with a password!
Otherwise, anyone who obtains access to the key will also be able to access the remote machines!
© 2017 All rights reserved.
Key-based authentication -2-
2. Copy the keys to their locations:
Private key – remains on local machine (~/.ssh/id_rsa)
Public key – copied to remote machine and added to the
~/.ssh/authorized_keys file
local]$ scp id_rsa.pub user@remote:~/
remote]$ cat id_rsa.pub >> ~/.ssh/authorized_keys
© 2017 All rights reserved.
© 2017 All rights reserved.
You might also like
- Introduction To Linux - Basic Commands & Environment - Linux-2No ratings yetIntroduction To Linux - Basic Commands & Environment - Linux-257 pages
- Johnson Grammar School: Kuntloor-HyderabadNo ratings yetJohnson Grammar School: Kuntloor-Hyderabad2 pages
- CPNT217 - 15. Basic Linux AdministrationNo ratings yetCPNT217 - 15. Basic Linux Administration42 pages
- SE Operating System Manual+Programs-rpc 10 ProgNo ratings yetSE Operating System Manual+Programs-rpc 10 Prog33 pages
- Red Hat System Administration I (RH124 Brief)No ratings yetRed Hat System Administration I (RH124 Brief)41 pages
- Introduction To Data Science and Python For DataNo ratings yetIntroduction To Data Science and Python For Data12 pages
- Linux Crash Course: Command Description Filesystem / PWD - ..No ratings yetLinux Crash Course: Command Description Filesystem / PWD - ..5 pages
- Lecture Summary: Course Introduction and SyllabusNo ratings yetLecture Summary: Course Introduction and Syllabus4 pages
- Finite Element Method For ElectromagneticsNo ratings yetFinite Element Method For Electromagnetics360 pages
- AWS-SOP - Creating ALB and Configuring Target Groups, Listeners and StickinessNo ratings yetAWS-SOP - Creating ALB and Configuring Target Groups, Listeners and Stickiness15 pages
- Introduction To Linux Operating Sysytem FinalNo ratings yetIntroduction To Linux Operating Sysytem Final32 pages
- Colour Dilution Alopecia in Doberman Pinschers With Blue or Fawn Coat Colours - A Study On The Incidence and Histopathology of This DiNo ratings yetColour Dilution Alopecia in Doberman Pinschers With Blue or Fawn Coat Colours - A Study On The Incidence and Histopathology of This Di10 pages
- Android-Controlled Pesticide Spraying RobotNo ratings yetAndroid-Controlled Pesticide Spraying Robot6 pages
- New Design of Intelligent Load Shedding Algorithm Based On Critical Line Overloads To Reduce Network Cascading Failure RisksNo ratings yetNew Design of Intelligent Load Shedding Algorithm Based On Critical Line Overloads To Reduce Network Cascading Failure Risks15 pages
- 2006-12-31: Overall Conclusion For The Year of 'Arise and Shine'No ratings yet2006-12-31: Overall Conclusion For The Year of 'Arise and Shine'6 pages
- Loop SMPTE - TST-B1 Until You Have Completed The QuestionsNo ratings yetLoop SMPTE - TST-B1 Until You Have Completed The Questions1 page