The Linux System Administrator's Guide

Version 0.9

Lars Wirzenius

<Email address removed by request>

Joanna Oja

<Current email address unknown>

Stephen Stafford

<[email protected]>

Alex Weeks

<[email protected]>

An introduction to system administration of a Linux system for novices.

Copyright 1993−−1998 Lars Wirzenius.

Copyright 1998−−2001 Joanna Oja.

Copyright 2001−−2003 Stephen Stafford.

Copyright 2003−−2004 Stephen Stafford & Alex Weeks.

Copyright 2004−−Present Alex Weeks.

Trademarks are owned by their owners. Permission is granted to copy,

distribute and/or modify this document under the terms of the GNU Free.
Documentation License, Version 1.2 or any later version published by the
Free Software Foundation; with no Invariant Sections, no Front−Cover Texts,
and no Back−Cover Texts. A copy of the license is included in the section
entitled "GNU Free Documentation License".

Table of contents

About this Book

1. Acknowledgments -1
1.1. Joanna’s Acknowledgments -1
1.2. Stephen’s acknowledgments -1
1.3. Alex’s acknowledgments -2
2. Revision History-2
3. Source and pre-formatted versions available -2
 4. Typographical conventions-3

Chapter 1. Introduction -4

1.1. Linux or GNU/Linux that is the question -5

1.2. Trademarks-5

Chapter 2. Overview of a Linux System-7

2.1.various parts of an operating system -7

2.2. Important parts of the kernel-7

2.3. major services in a UNIX system -8

2.3.1. init-9

2.3.2. logins from terminals-9

2.3.3.- syslog-9

2.3.4. periodic command execution: cron and at-10

2.3.5. Graphical user interface -10

2.3.6. networking -10

2.3.7. Network logins-11

2.3.8. Network file systems-11

2.3.9. Mail-11

2.3.10. Printing -12

2.3.11. The filesystem layout-12

Chapter 3. Overview of the directory tree-13

3.1. Background -13

3.2. the root filesystem -14

3.3. The/etc directory -15

3.4. The/dev directory -17

3.5. The/usr filesystem -19

3.6. the/var filesystem -19

3.7. the/Proc filesystem -20

Chapter 4. Hardware, devices and tools-22

4.1. Hardware utilities -22

4.1.1. The Makedev script-22
4.1.2. The mknod command-22
4.1.3. The lspci command -23
4.1.4. The lsusb command -23
4.1.5. The lsraid command -23
4.1.6. The Israid command-23
4.1.7. The hdparm command23
4.1.8. More hardware resources -23
4.2. Kernel modules-23
4.2.1. Ismod-24
4.2.2. Insmod-24
4.2.3. Depmod-24
4.2.4. Rmmod-24
4.2.5. Modprobe-24

Chapter 5. Using disks and other storage media

5.1. two kinds of devices- 25

5.2. hard disks- 26

5.3. storage areas networks- drafts-28

5.4. network attached storage -draft-28

5.4.1. NFS-29

5.4.2. CIFS-29

5.5. Floppies-29

5.6. CD-ROMS-30

5.7. Tapes-31

5.8. formatting -31

5.9. Partitions- 32

5.9.1. The MBR, boot sectors and partition table-33

5.9.2. extended and logical partitions -33

5.9.3. Partition types-34

5.9.4. partitioning a hard disk-35

5.9.5. device files and partitions-36

5.10. filesystems-36

5.10.1. what are filesystems? -36

5.10.2. filesystems galore-37

5.10.3. which filesystems should be used?- 39

5.10.4. creating a filesystem -39

5.10.5. filesystem block size -40

5.10.6. filesystem comparison-41

5.10.7. mounting and unmounting- 42

5.10.8. filesystem security-45

5.10.9. checking filesystem integrity with fsck-45

5.10.10. checking for disk errors with bad blocks-46

5.10.11. fighting fragmentation? -46

5.10.12. other tools for all filesystems-47

5.10.13. other tools for the ext2/ext3 filesystem -47

5.11. disks without filesystems-48

5.12. Allocating disk space-49

5.12.1. partitioning schemes-49

5.12.2. logical volume manager(LVM)- 50

5.12.3. space requirements -50

5.12.4. examples of hard disk allocation-50

5.12.5. adding More disk space for Linux-51

5.12.6. tips for saving disk space-51

Chapter 6. Memory Management -52

6.1. what is virtual memory?-52

6.2. creating a swap space-52

6.3. using a swap space-53

6.4. sharing swap spaces with other operating systems-54

6.5. allocating a swap space-55

6.6. the buffer cache-56

Chapter 7. System monitoring-58

7.1. system resources -58

7.1.1. the top command -58

7.1.2. the iostat command- 59

7.1.3. the ps command-60

7.1.4. the vmstat command -61

7.1.5. the lsof command -61

7.1.6. finding more utilities -62

7.2. filesystem usage-62

7.2.1. the df command -62

7.2.2. the du command -62

7.2.3. quotas-63

7.3. monitoring users-63

7.3.1. the who command -63

7.3.2. the ps command- again!- 64

7.3.3. the w command-64

7.3.4. the skill command -64

7.3.5. nice and renice -64

Chapter 8. Boots and shutdowns-65

8.1. An overview of boots and shutdowns-65

8.2. The boot process in closure look-65

8.2.1. A word about bootloaders- 67

8.3. More about shutdowns-67

8.4. rebooting-68

8.5. single user mode-69

8.6. emergency boot floppies-69

Chapter 9. Init-70

9.1. init comes first-70

9.2. configuring init to start getty: the/etc/inittab file-70

9.3. Run levels-71

9.4. special configuration in/etc/inittab-73

9.5. booting in single user mode-73

Chapter 10. Logging In and out-75

10.1. logins via terminals- 75

10.2. logins via the network -76

10.3. what login does-77

10.4. X and xdm- 78

10.5. Access control -78

10.6. shell startup -78

Chapter 11. Managing user and accounts-79

11.1. what’s an account -79

11.2. creating a user-79

11.2.1. /etc/password and other informative files -79

11.2.2. picking numeric user and group ids-80

11.2.3. initial environment:/etc/skel-80

11.2.4. creating a user by hand-80

11.3. changing user properties -81

11.4. removing a user-81

11.5. disabling a user temporarily – 82

Chapter 12. Backups-83

12.1. on the importance of being backed up -83

12.2. selecting the backup mediums -83

12.3. selecting the backup tool-84

12.4. simple backups-85

12.4.1. making backups with tar-85

12.4.2. restoring files with tar-86

12.5. multilevel backups-87

12.6. what to back up-88

12.7. compressed backups- 89

Chapter 13. Task Automation- to be added-90

Chapter 14. Keeping time-91

14.1. the concept of local time-91

14.2. the hardware and software clocks-92

14.3. showing and setting time-92

14.4. when the clock is wrong-93

14.5. NTP- Network time protocol -93

14.6. Basic NTP configuration -94

14.7. NTP toolkit-95

14.8. some known NTP servers-97

14.9. NTP links-97

Chapter 15. System logs- To be Added-98

Chapter 16. System updates- To be Added-99

Chapter 17. The Linux kernel source-100

Chapter 18. Finding help-101

18.1. newsgroups and mailing lists-101

18.1.1. finding the right forum-101

Chapter 18. Finding help

18.1.2. before you post-101

18.1.3. writing your post -101

18.1.4. formatting your post -102

18.1.5. follow up-102

18.1.6. more information -102

18.2. IRC-102

18.2.1. colours-103

18.2.2. be polite -103

18.2.3. type properly in English -103

18.2.4. Port scanning -103

18.2.5. keep in the channel -103

18.2.6. stay on topic-104

18.2.7. CTCP’s-104

18.2.8. hacking, cracking, phreaking, warezing-104

18.2.9. Round up-104

18.2.10. Further reading-104

Appendix A. GNU free documentation license -105

A.1. PREAMBLE-105

A.2. APPLICATION AND DEFINITIONS-105

A.3. VERBATIM COPYING-106

A.4. COPYING IN QUANTITY -106

A.5. MODIFICATIONS-107

A.6. COMBINING DOCUMENTS -108

A.7. COLLECTIONS OF DOCUMENTS -109

A.8. AGGREGATION WITH INDEPENDENT WORKS-109

A.10. TERMINATION -109

A.11. FUTURE REVISIONS OF THIS LICENSE -110

A.12. APPENDUM: how to use this license for your documents -110

Glossary (Draft, but not for long hopefully)-111

Index- draft-115

A-115

B-115

C-115

D-116

E-117

F-117

G-120

H-120

I-120

K-120

L-121

M-121

N-121

O-121

Glossary (draft, but not for long hopefully)

P-121

R-122

S-122

T-122

V-122

W-122

About this Book

“ two things are infinite, the universe and human stupidity, and I’m not sure
about former.” Albert Einstein
1. Acknowledgments
1.1. Joanna’s acknowledgments

Many people have helped me with this book, directly or indirectly. I would
like to especially thank Matt Welsh for inspiration and LDP leadership,
Andy Oram for getting me to work again with much−valued feedback,
Olaf Kirch for showing me that it can be done, and Adam Richter at
Yggdrasil and others for showing me that other people can find it
interesting as well. Stephen Tweedie, H. Peter Anvin, Remy Card,
Theodore Ts’o, and Stephen Tweedie have let me borrow their work (and
thus make the book look thicker and much more impressive): a
comparison between the xia and ext2 filesystems, the device list and a
description of the ext2 filesystem. These aren’t part of the book any more.
I am most grateful for this, and very apologetic for the earlier versions
that sometimes lacked proper attribution. In addition, I would like to thank
Mark Komarinski for sending his material in 1993 and the many system
administration columns in Linux Journal. They are quite informative and
inspirational.

Many useful comments have been sent by a large number of people. My

miniature black hole of an archive doesn’t let me find all their names, but
some of them are, in alphabetical order: Paul Caprioli, Ales
Cepek,Marie−France Declerfayt, Dave Dobson, Olaf Flebbe, Helmut Geyer,
Larry Greenfield and his father, Stephen Harris, Jyrki Havia, Jim Haynes,
York Lam, Timothy Andrew Lister, Jim Lynch, Michael J. Micek, Jacob Navia,
Dan Poirier, Daniel Quinlan, Jouni K Seppänen, Philippe Steindl, G.B.
Stotte. My apologies to anyone I have forgotten.

1.2. Stephen’s acknowledgments

I would like to thank Lars and Joanna for their hard work on the guide.

In a guide like this one there are likely to be at least some minor
inaccuracies. And there are almost certainly going to be sections that
become out of date from time to time. If you notice any of this then
please let me know by sending me an email to:
[email protected]. I will take virtually any form of input (diffs,
just plain text, html, whatever), I am in no way above allowing others to
help me maintain such a large text as this.

Many thanks to Helen Topping Shaw for getting the red pen out and
making the text far better than it would otherwise have been. Also thanks
are due just for being wonderful.
 1.3. Alex’s Acknowledgments

I would like to thank Lars, Joanna, and Stephen for all the great work that
they have done on this document over the years. I only hope that my
contribution will be worthy of continuing the work they started.

Like the previous maintainers, I openly welcome any comments,

suggestions, complains, corrections, or any other form of feedback you
may have. This document can only benefit from the suggestions of those
who use it.

There have been many people who have helped me on my journey

through the “Windows−Free” world, the person I feel I need to thank the
most is my first true UN*X mentor, Mike Velasco. Back in a time before
SCO became a “dirty word”, Mike helped me on the path of tar’s, cpio’s,
and many, many man pages. Thanks Mike! You are the ‘Sofa King’.

2.Revision History

Revision History

Revision 0.7 2001−12−03 Revised by: SS

Revision 0.8 2003−11−18 Revised by:AW

1.Added a section on NTP

2.Cleaned some SGML

3.Added ext3 to the filesystem section

Revision 0.9 Revised by: AW

1.Cleaned some SGML code, changed doctype to lds.dsl, and added id

tags

2.Updated section on filesystem types, and Filesystem comparison

3.Updated partition type section

4.Updated section on creating partitions

5.Wrote section on Logical Volume Manager (LVM)

6.Updated section on space allocation

7.Added chapter on System Monitoring

8.Added more command line utilities

9.Verified Device list

10.Modified email address for Authors

11.Added references to more in−depth documents where applicable

12.Added notes on upcoming sections

13.Indexed chapters 1 – 4, & part of 5

14.Updated Misc Information throughout the book

3.Source and pre−formatted versions available

The source code and other machine readable formats of this book can be
found on the Internet via anonymous FTP at the Linux Documentation Project
home page http://www.tldp.org/, or at the home page of this book at
http://www.draxeman/sag.html. This book is available in at least it’s SGML
source, as well as, HTML and PDF formats. Other formats may be available.

4.Typographical Conventions

Throughout this book, I have tried to use uniform typographical conventions.

Hopefully they aid readability. If you can suggest any improvements please
contact me.

Filenames are expressed as: /usr/share/doc/foo.

Command names are expressed as: fsck

Email addresses are expressed as: [email protected]

URLs are expressed as: http://www.tldp.org

I will add to this section as things come up whilst editing. If you notice
anything that should be added then please let me know.

Chapter 1. Introduction

"In the beginning, the file was without form, and void; and emptiness was
upon the face of the bits. And the Fingers of the Author moved upon the face
of the keyboard. And the Author said, Let there be words, and there were
words."

The Linux System Administrator's Guide, describes the system administration

aspects of using Linux. It is intended for people who know next to nothing
about system administration (those saying what is it?), but who have already
mastered at least the basics of normal usage. This manual doesn't tell you
how to install Linux; that is described in the Installation and Getting Started
document. See below for more information about Linux manuals.

System administration covers all the things that you have to do to keep a
computer system in usable order. It includes things like backing up files (and
restoring them if necessary), installing new programs, creating accounts for
users (and deleting them when no longer needed), making certain that the
filesystem is not corrupted, and so on. If a computer were, say, a house,
system administration would be called maintenance, and would include
cleaning, fixing broken windows, and other such things.

The structure of this manual is such that many of the chapters should be
usable independently, so if you need information about backups, for
example, you can read just that chapter. However, this manual is first and
foremost a tutorial and can be read sequentially or as a whole.

This manual is not intended to be used completely independently. Plenty of

the rest of the Linux documentation is also important for system
administrators. After all, a system administrator is just a user with special
privileges and duties. Very useful resources are the manual pages, which
should always be consulted when you are not familiar with a command. If
you do not know which command you need, then the apropos command can
be used. Consult its manual page for more details.

While this manual is targeted at Linux, a general principle has been that it
should be useful with other UNIX based operating systems as well.
Unfortunately, since there is so much variance between different versions of
UNIX in general, and in system administration in particular, there is little
hope to cover all variants. Even covering all possibilities for Linux is difficult,
due to the nature of its development.

There is no one official Linux distribution, so different people have different

setups and many people have a setup they have built up themselves. This
book is not targeted at any one distribution. Distributions can and do vary
considerably. When possible, differences have been noted and alternatives
given. For a list of distributions and some of their differences see
http://en.wikipedia.org/wiki/Comparison_of_Linux_distributions.

In trying to describe how things work, rather than just listing five easy steps''
for each task, there is much information here that is not necessary for
everyone, but those parts are marked as such and can be skipped if you use
a preconfigured system. Reading everything will, naturally, increase your
understanding of the system and should make using and administering it
more productive.

Understanding is the key to success with Linux. This book could just provide
recipes, but what would you do when confronted by a problem this book had
no recipe for? If the book can provide understanding, then recipes are not
required. The answers will be self evident.

Like all other Linux related development, the work to write this manual was
done on a volunteer basis: I did it because I thought it might be fun and
because I felt it should be done. However, like all volunteer work, there is a
limit to how much time, knowledge and experience people have. This means
that the manual is not necessarily as good as it would be if a wizard had
been paid handsomely to write it and had spent millennia to perfect it. Be
warned.

One particular point where corners have been cut is that many things that
are already well documented in other freely available manuals are not
always covered here. This applies especially to program specific
documentation, such as all the details of using mkfs. Only the purpose of the
program and as much of its usage as is necessary for the purposes of this
manual is described. For further information, consult these other manuals.
Usually, all of the referred to documentation is part of the full Linux
documentation set.

1.1. Linux or GNU/Linux, that is the question.

Many people feel that Linux should really be called GNU/Linux. This is
because Linux is only the kernel, not the applications that run on it. Most of
the basic command line utilities were written by the Free Software
Foundation while developing their GNU operating system. Among those
utilities are some of the most basic commands like cp, mv lsof, and dd.

In a nutshell, what happened was, the FSF started developing GNU by writing
things like compliers, C libraries, and basic command line utilities before the
kernel. Linus Torvalds, started Linux by writing the Linux kernel first and
using applications written for GNU.

I do not feel that this is the proper forum to debate what name people should
use when referring to Linux. I mention it here, because I feel it is important
to understand the relationship between GNU and Linux, and to also explain
why some Linux is sometimes referred to as GNU/Linux. The document will
be simply referring to it as Linux.
GNU's side of the issue is discussed on their website:

The relationship − http://www.gnu.org/gnu/linux−and−gnu.html

Why Linux should be GNU/Linux −

http://www.gnu.org/gnu/why−gnu−linux.html

GNU/Linux FAQ's − http://www.gnu.org/gnu/gnu−linux−faq.html Here are

some Alternate views: http://librenix.com/?inode=2312
http://www.topology.org/linux/lingl.html
http://atulchitnis.net/writings/gnulinux.php

1.2. Trademarks

Microsoft, Windows, Windows NT, Windows 2000, and Windows XP are

trademarks and/or registered trademarks of Microsoft Corporation.

Red Hat is a trademark of Red Hat, Inc., in the United States and other
countries.

SuSE is a trademark of Novell.

Linux is a registered trademark of Linus Torvalds.

UNIX is a registered trademark in the United States and other countries,

licensed exclusively through X/Open Company Ltd.

GNU is a registered trademark of the Free Software Foundation.

Other product names mentioned herein may be trademarks and/or registered

trademarks of their respective companies.

Chapter 2. Overview of a Linux System

"God saw everything that he had made, and saw that it was very good. " −−
Bible King James Version. Genesis 1:31

This chapter gives an overview of a Linux system. First, the major services
provided by the operating system are described. Then, the programs that
implement these services are described with a considerable lack of detail.
The purpose of this chapter is to give an understanding of the system as a
whole, so that each part is described in detail elsewhere.

2.1. Various parts of an operating system

UNIX and 'UNIX−like' operating systems (such as Linux) consist of a kernel

and some system programs. There are also some application programs for
doing work. The kernel is the heart of the operating system. In fact, it is often
mistakenly considered to be the operating system itself, but it is not. An
operating system provides provides many more services than a plain kernel.

It keeps track of files on the disk, starts programs and runs them
concurrently, assigns memory and other resources to various processes,
receives packets from and sends packets to the network, and so on. The
kernel does very little by itself, but it provides tools with which all services
can be built. It also prevents anyone from accessing the hardware directly,
forcing everyone to use the tools it provides. This way the kernel provides
some protection for users from each other. The tools provided by the kernel
are used via system calls. See manual page section 2 for more information
on these.

The system programs use the tools provided by the kernel to implement the
various services required from an operating system. System programs, and
all other programs, run `on top of the kernel', in what is called the user
mode. The difference between system and application programs is one of
intent: applications are intended for getting useful things done (or for
playing, if it happens to be a game), whereas system programs are needed
to get the system working. A word processor is an application; mount is a
system program. The difference is often somewhat blurry, however, and is
important only to compulsive categorizers.

An operating system can also contain compilers and their corresponding

libraries (GCC and the C library in particular under Linux), although not all
programming languages need be part of the operating system.
Documentation, and sometimes even games, can also be part of it.
Traditionally, the operating system has been defined by the contents of the
installation tape or disks; with Linux it is not as clear since it is spread all
over the FTP sites of the world.

2.2. Important parts of the kernel

The Linux kernel consists of several important parts: process management,

memory management, hardware device drivers, filesystem drivers, network
management, and various other bits and pieces.

Probably the most important parts of the kernel (nothing else works without
them) are memory management and process management. Memory
management takes care of assigning memory areas and swap space areas to
processes, parts of the kernel, and for the buffer cache. Process
management creates processes, and implements multitasking by switching
the active process on the processor.
At the lowest level, the kernel contains a hardware device driver for each
kind of hardware it supports. Since the world is full of different kinds of
hardware, the number of hardware device drivers is large. There are often
many otherwise similar pieces of hardware that differ in how they are
controlled by software. The similarities make it possible to have general
classes of drivers that support similar operations; each member of the class
has the same interface to the rest of the kernel but differs in what it needs to
do to implement them. For example, all disk drivers look alike to the rest of
the kernel, i.e., they all have operations like initialize the drive', read sector
N', and write sector N'.

Some software services provided by the kernel itself have similar properties,
and can therefore be abstracted into classes. For example, the various
network protocols have been abstracted into one programming interface, the
BSD socket library. Another example is the virtual filesystem (VFS) layer that
abstracts the filesystem operations away from their implementation. Each
filesystem type provides an implementation of each filesystem operation.
When some entity tries to use a filesystem, the request goes via the VFS,
which routes the request to the proper filesystem driver.

A more in−depth discussion of kernel internals can be found at

http://www.tldp.org/LDP/lki/index.html. This document was written for the 2.4
kernel. When I find one for the 2.6 kernel, I will list it here.

2.3. Major services in a UNIX system

This section describes some of the more important UNIX services, but
without much detail. They are described more thoroughly in later chapters.

2.3.1. init

The single most important service in a UNIX system is provided by init init is
started as the first process of every UNIX system, as the last thing the kernel
does when it boots. When init starts, it continues the boot process by doing
various startup chores (checking and mounting filesystems, starting
daemons, etc).

The exact list of things that init does depends on which flavor it is; there are
several to choose from. init usually provides the concept of single user mode,
in which no one can log in and root uses a shell at the console; the usual
mode is called multiuser mode. Some flavors generalize this as run levels;
single and multiuser modes are considered to be two run levels, and there
can be additional ones as well, for example, to run X on the console.
Linux allows for up to 10 runlevels, 0−9, but usually only some of these are
defined by default. Runlevel 0 is defined as ``system halt''. Runlevel 1 is
defined as ``single user mode''. Runlevel 3 is defined as "multi user"
because it is the runlevel that the system boot into under normal day to day
conditions. Runlevel 5 is typically the same as 3 except that a GUI gets
started also. Runlevel 6 is defined as ``system reboot''. Other runlevels are
dependent on how your particular distribution has defined them, and they
vary significantly between distributions. Looking at the contents of
/etc/inittab usually will give some hint what the predefined runlevels are and
what they have been defined as.

In normal operation, init makes sure getty is working (to allow users to log in)
and to adopt orphan processes (processes whose parent has died; in UNIX all
processes must be in a single tree, so orphans must be adopted).

When the system is shut down, it is init that is in charge of killing all other
processes, unmounting all filesystems and stopping the processor, along with
anything else it has been configured to do.

2.3.2. Logins from terminals

Logins from terminals (via serial lines) and the console (when not running X)
are provided by the getty program. init starts a separate instance of getty for
each terminal upon which logins are to be allowed. getty reads the username
and runs the loginprogram, which reads the password. If the username and
password are correct, login runs the shell. When the shell terminates, i.e., the
user logs out, or when login terminated because the username and password
didn't match, init notices this and starts a new instance of getty. The kernel
has no notion of logins, this is all handled by the system programs.

2.3.3. Syslog

The kernel and many system programs produce error, warning, and other
messages. It is often important that these messages can be viewed later,
even much later, so they should be written to a file. The program doing this
is syslog . It can be configured to sort the messages to different files
according to writer or degree of importance. For example, kernel messages
are often directed to a separate file from the others, since kernel messages
are often more important and need to be read regularly to spot
problems.Chapter 15 will provide more on this.

2.3.4. Periodic command execution: cron and at

Both users and system administrators often need to run commands
periodically. For example, the system administrator might want to run a
command to clean the directories with temporary files (/tmp and /var/tmp)
from old files, to keep the disks from filling up, since not all programs clean
up after themselves correctly.

The cron service is set up to do this. Each user can have a crontab file, where
she lists the commands she wishes to execute and the times they should be
executed. The cron daemon takes care of starting the commands when
specified.

The at service is similar to cron, but it is once only: the command is executed
at the given time, but it is not repeated.

We will go more into this later. See the manual pages cron(1), crontab(1),
crontab(5), at(1) and atd(8) for more in depth information.Chapter 13 will
cover this.

2.3.5. Graphical user interface

UNIX and Linux don't incorporate the user interface into the kernel; instead,
they let it be implemented by user level programs. This applies for both text
mode and graphical environments.

This arrangement makes the system more flexible, but has the disadvantage
that it is simple to implement a different user interface for each program,
making the system harder to learn.

The graphical environment primarily used with Linux is called the X Window
System (X for short). X also does not implement a user interface; it only
implements a window system, i.e., tools with which a graphical user interface
can be implemented. Some popular window managers are: fvwm , icewm ,
blackbox , and windowmaker . There are also two popular desktop managers,
KDE and Gnome.

2.3.6. Networking

Networking is the act of connecting two or more computers so that they can
communicate with each other. The actual methods of connecting and
communicating are slightly complicated, but the end result is very useful.

UNIX operating systems have many networking features. Most basic services
(filesystems, printing, backups, etc) can be done over the network. This can
make system administration easier, since it allows centralized
administration, while still reaping in the benefits of microcomputing and
distributed computing, such as lower costs and better fault tolerance.

However, this book merely glances at networking; see the Linux Network
Administrators' Guide http://www.tldp.org/LDP/nag2/index.html for more
information, including a basic description of how networks operate.

2.3.7. Network logins

Network logins work a little differently than normal logins. For each person
logging in via the network there is a separate virtual network connection,
and there can be any number of these depending on the available
bandwidth. It is therefore not possible to run a separate getty for each
possible virtual connection. There are also several different ways to log in via
a network, telnet and ssh being the major ones in TCP/IP networks.

These days many Linux system administrators consider telnet and rlogin to
be insecure and prefer ssh, the ``secure shell'', which encrypts traffic going
over the network, thereby making it far less likely that the malicious can
``sniff'' your connection and gain sensitive data like usernames and
passwords. It is highly recommended you use ssh rather than telnet or rlogin.

Network logins have, instead of a herd of gettys, a single daemon per way of
logging in (telnet and ssh have separate daemons) that listens for all
incoming login attempts. When it notices one, it starts a new instance of
itself to handle that single attempt; the original instance continues to listen
for other attempts. The new instance works similarly to getty.

2.3.8. Network file systems

One of the more useful things that can be done with networking services is
sharing files via a network file system. Depending on your network this could
be done over the Network File System (NFS), or over the Common Internet
File System (CIFS). NFS is typically a 'UNIX' based service. In Linux, NFS is
supported by the kernel. CIFS however is not. In Linux, CIFS is supported by
Samba http://www.samba.org.

With a network file system any file operations done by a program on one
machine are sent over the network to another computer. This fools the
program to think that all the files on the other computer are actually on the
computer the program is running on. This makes information sharing
extremely simple, since it requires no modifications to programs.

This will be covered in more detail in Section 5.4.

2.3.9. Mail

Electronic mail is the most popularly used method for communicating via
computer. An electronic letter is stored in a file using a special format, and
special mail programs are used to send and read the letters.

Each user has an incoming mailbox (a file in the special format), where all
new mail is stored. When someone sends mail, the mail program locates the
receiver's mailbox and appends the letter to the mailbox file. If the receiver's
mailbox is in another machine, the letter is sent to the other machine, which
delivers it to the mailbox as it best sees fit.

The mail system consists of many programs. The delivery of mail to local or
remote mailboxes is done by one program (the mail transfer agent (MTA) ,
e.g., sendmail or postfix ), while the programs users use are many and varied
(mail user agent (MUA) , e.g., pine , or evolution . The mailboxes are usually
stored in /var/spool/mail until the user's MUA retrieves them.

For more information on setting up and running mail services you can read
the Mail Administrator HOWTO at
http://www.tldp.org/HOWTO/Mail−Administrator−HOWTO.html, or visit the
sendmail or postfix's website. http://www.sendmail.org/, or
http://www.postfix.org/ .

2.3.10. Printing

Only one person can use a printer at one time, but it is uneconomical not to
share printers between users. The printer is therefore managed by software
that implements a print queue: all print jobs are put into a queue and
whenever the printer is done with one job, the next one is sent to it
automatically. This relieves the users from organizing the print queue and
fighting over control of the printer. Instead, they form a new queue at the
printer, waiting for their printouts, since no one ever seems to be able to get
the queue software to know exactly when anyone's printout is really finished.
This is a great boost to intra−office social relations.

The print queue software also spools the printouts on disk, i.e., the text is
kept in a file while the job is in the queue. This allows an application program
to spit out the print jobs quickly to the print queue software; the application
does not have to wait until the job is actually printed to continue. This is
really convenient, since it allows one to print out one version, and not have
to wait for it to be printed before one can make a completely revised new
version.You can refer to the Printing−HOWTO located at
http://www.tldp.org/HOWTO/Printing−HOWTO/index.html for more help in
setting up printers.

2.3.11. The filesystem layout

The filesystem is divided into many parts; usually along the lines of a root
filesystem with /bin , /lib , /etc , /dev , and a few others; a /usr filesystem with
programs and unchanging data; /var filesystem with changing data (such as
log files); and a /home for everyone's personal files. Depending on the
hardware configuration and the decisions of the system administrator, the
division can be different; it can even be all in one filesystem.

Chapter 3 describes the filesystem layout in some little detail; the Filesystem
Hierarchy Standard . covers it in somewhat more detail. This can be found on
the web at: http://www.pathname.com/fhs/

Chapter 3. Overview of the Directory Tree

" Two days later, there was Pooh, sitting on his branch, dangling his legs, and
there, beside him, were four pots of honey..." (A.A. Milne)

This chapter describes the important parts of a standard Linux directory tree,
based on the Filesystem Hierarchy Standard . It outlines the normal way of
breaking the directory tree into separate filesystems with different purposes
and gives the motivation behind this particular split. Not all Linux
distributions follow this standard slavishly, but it is generic enough to give
you an overview.

3.1. Background

This chapter is loosely based on the Filesystems Hierarchy Standard (FHS).

version 2.1, which attempts to set a standard for how the directory tree in a
Linux system is organized. Such a standard has the advantage that it will be
easier to write or port software for Linux, and to administer Linux machines,
since everything should be in standardized places. There is no authority
behind the standard that forces anyone to comply with it, but it has gained
the support of many Linux distributions. It is not a good idea to break with
the FHS without very compelling reasons. The FHS attempts to follow Unix
tradition and current trends, making Linux systems familiar to those with
experience with other Unix systems, and vice versa.

This chapter is not as detailed as the FHS. A system administrator should

also read the full FHS for a complete understanding.
This chapter does not explain all files in detail. The intention is not to
describe every file, but to give an overview of the system from a filesystem
point of view. Further information on each file is available elsewhere in this
manual or in the Linux manual pages.

The full directory tree is intended to be breakable into smaller parts, each
capable of being on its own disk or partition, to accommodate to disk size
limits and to ease backup and other system administration tasks. The major
parts are the root (/ ), /usr , /var , and /home filesystems (see Figure 3−1).
Each part has a different purpose. The directory tree has been designed so
that it works well in a network of Linux machines which may share some
parts of the filesystems over a read−only device (e.g., a CD−ROM), or over
the network with NFS. Probably the most important parts of the kernel
(nothing else works without them) are memory management and process
management. Memory management takes care of assigning memory areas
and swap space areas to processes, parts of the kernel, and for the buffer
cache. Process management creates processes, and implements multitasking
by switching the active process on the processor.

At the lowest level, the kernel contains a hardware device driver for each
kind of hardware it supports. Since the world is full of different kinds of
hardware, the number of hardware device drivers is large. There are often
many otherwise similar pieces of hardware that differ in how they are
controlled by software. The similarities make it possible to have general
classes of drivers that support similar operations; each member of the class
has the same interface to the rest of the kernel but differs in what it needs to
do to implement them. For example, all disk drivers look alike to the rest of
the kernel, i.e., they all have operations like `initialize the drive', `read sector
N', and `write sector N'.

Some software services provided by the kernel itself have similar properties,
and can therefore be abstracted into classes. For example, the various
network protocols have been abstracted into one programming interface, the
BSD socket library. Another example is the virtual filesystem (VFS) layer that
abstracts the filesystem operations away from their implementation. Each
filesystem type provides an implementation of each filesystem operation.
When some entity tries to use a filesystem, the request goes via the VFS,
which routes the request to the proper filesystem driver.

A more in−depth discussion of kernel internals can be found at

http://www.tldp.org/LDP/lki/index.html. This document was written for the 2.4
kernel. When I find one for the 2.6 kernel, I will list it here.
2.3. Major services in a UNIX system

This section describes some of the more important UNIX services, but
without much detail. They are described more thoroughly in later chapters.

2.3.1. init

The single most important service in a UNIX system is provided by init init is
started as the first process of every UNIX system, as the last thing the kernel
does when it boots. When init starts, it continues the boot process by doing
various startup chores (checking and mounting filesystems, starting
daemons, etc).

The exact list of things that init does depends on which flavor it is; there are
several to choose from. init usually provides the concept of single user mode,
in which no one can log in and root uses a shell at the console; the usual
mode is called multiuser mode. Some flavors generalize this as run levels;
single and multiuser modes are considered to be two run levels, and there
can be additional ones as well, for example, to run X on the console.

Linux allows for up to 10 runlevels, 0−9, but usually only some of these are
defined by default. Runlevel 0 is defined as ``system halt''. Runlevel 1 is
defined as ``single user mode''. Runlevel 3 is defined as "multi user"
because it is the runlevel that the system boot into under normal day to day
conditions. Runlevel 5 is typically the same as 3 except that a GUI gets
started also. Runlevel 6 is defined as ``system reboot''. Other runlevels are
dependent on how your particular distribution has defined them, and they
vary significantly between distributions. Looking at the contents of
/etc/inittab usually will give some hint what the predefined runlevels are and
what they have been defined as.

In normal operation, init makes sure getty is working (to allow users to log in)
and to adopt orphan processes (processes whose parent has died; in UNIX all
processes must be in a single tree, so orphans must be adopted).

When the system is shut down, it is init that is in charge of killing all other
processes, unmounting all filesystems and stopping the processor, along with
anything else it has been configured to do.
2.3.2. Logins from terminals

Logins from terminals (via serial lines) and the console (when not running X)
are provided by the getty program. init starts a separate instance of getty for
each terminal upon which logins are to be allowed. getty reads the username
and runs the loginprogram, which reads the password. If the username and
password are correct, login runs the shell. When the shell terminates, i.e., the
user logs out, or when login terminated because the username and password
didn't match, init notices this and starts a new instance of getty. The kernel
has no notion of logins, this is all handled by the system programs.

2.3.3. Syslog

The kernel and many system programs produce error, warning, and other
messages. It is often important that these messages can be viewed later,
even much later, so they should be written to a file. The program doing this
is syslog . It can be configured to sort the messages to different files
according to writer or degree of importance. For example, kernel messages
are often directed to a separate file from the others, since kernel messages
are often more important and need to be read regularly to spot problems.

Chapter 15 will provide more on this.

2.3.4. Periodic command execution: cron and at

Both users and system administrators often need to run commands

periodically. For example, the system administrator might want to run a
command to clean the directories with temporary files (/tmp and

/var/tmp) from old files, to keep the disks from filling up, since not all
programs clean up after themselves correctly.

The cron service is set up to do this. Each user can have a crontab file, where
she lists the commands she wishes to execute and the times they should be
executed. The cron daemon takes care of starting the commands when
specified.

The at service is similar to cron, but it is once only: the command is executed
at the given time, but it is not repeated.

We will go more into this later. See the manual pages cron(1), crontab(1),
crontab(5), at(1) and atd(8) for more in depth information.
Chapter 13 will cover this.

2.3.5. Graphical user interface

UNIX and Linux don't incorporate the user interface into the kernel; instead,
they let it be implemented by user level programs. This applies for both text
mode and graphical environments.

This arrangement makes the system more flexible, but has the disadvantage
that it is simple to implement a different user interface for each program,
making the system harder to learn.

The graphical environment primarily used with Linux is called the X Window
System (X for short). X also does not implement a user interface; it only
implements a window system, i.e., tools with which a graphical user interface
can be implemented. Some popular window managers are: fvwm , icewm ,
blackbox , and windowmaker . There are also two popular desktop managers,
KDE and Gnome.

2.3.6. Networking

Networking is the act of connecting two or more computers so that they can
communicate with each other. The actual methods of connecting and
communicating are slightly complicated, but the end result is very useful.

UNIX operating systems have many networking features. Most basic services
(filesystems, printing, backups, etc) can be done over the network. This can
make system administration easier, since it allows centralized
administration, while still reaping in the benefits of microcomputing and
distributed computing, such as lower costs and better fault tolerance.

However, this book merely glances at networking; see the Linux Network
Administrators' Guide http://www.tldp.org/LDP/nag2/index.html for more
information, including a basic description of how networks operate.

2.3.7. Network logins

Network logins work a little differently than normal logins. For each person
logging in via the network there is a separate virtual network connection,
and there can be any number of these depending on the available
bandwidth. It is therefore not possible to run a separate getty for each
possible virtual connection. There are also several different ways to log in via
a network, telnet and ssh being the major ones in TCP/IP networks.

These days many Linux system administrators consider telnet and rlogin to
be insecure and prefer ssh, the ``secure shell'', which encrypts traffic going
over the network, thereby making it far less likely that the malicious can
``sniff'' your connection and gain sensitive data like usernames and
passwords. It is highly recommended you use ssh rather than telnet or rlogin.

Network logins have, instead of a herd of gettys, a single daemon per way of
logging in (telnet and ssh have separate daemons) that listens for all
incoming login attempts. When it notices one, it starts a new instance of
itself to handle that single attempt; the original instance continues to listen
for other attempts. The new instance works similarly to getty.

2.3.8. Network file systems

One of the more useful things that can be done with networking services is
sharing files via a network file system. Depending on your network this could
be done over the Network File System (NFS), or over the Common Internet
File System (CIFS). NFS is typically a 'UNIX' based service. In Linux, NFS is
supported by the kernel. CIFS however is not. In Linux, CIFS is supported by
Samba http://www.samba.org.

With a network file system any file operations done by a program on one
machine are sent over the network to another computer. This fools the
program to think that all the files on the other computer are actually on the
computer the program is running on. This makes information sharing
extremely simple, since it requires no modifications to programs.

This will be covered in more detail in Section 5.4.

2.3.9. Mail

Electronic mail is the most popularly used method for communicating via
computer. An electronic letter is stored in a file using a special format, and
special mail programs are used to send and read the letters.

Each user has an incoming mailbox (a file in the special format), where all
new mail is stored. When someone sends mail, the mail program locates the
receiver's mailbox and appends the letter to the mailbox file. If the receiver's
mailbox is in another machine, the letter is sent to the other machine, which
delivers it to the mailbox as it best sees fit.

The mail system consists of many programs. The delivery of mail to local or
remote mailboxes is done by one program (the mail transfer agent (MTA) ,
e.g., sendmail or postfix ), while the programs users use are many and varied
(mail user agent (MUA) , e.g., pine , or evolution . The mailboxes are usually
stored in /var/spool/mail until the user's MUA retrieves them.

For more information on setting up and running mail services you can read
the Mail Administrator HOWTO at
http://www.tldp.org/HOWTO/Mail−Administrator−HOWTO.html, or visit the
sendmail or postfix's website. http://www.sendmail.org/, or
http://www.postfix.org/ .

2.3.10. Printing

Only one person can use a printer at one time, but it is uneconomical not to
share printers between users. The printer is therefore managed by software
that implements a print queue: all print jobs are put into a queue and
whenever the printer is done with one job, the next one is sent to it
automatically. This relieves the users from organizing the print queue and
fighting over control of the printer. Instead, they form a new queue at the
printer, waiting for their printouts, since no one ever seems to be able to get
the queue software to know exactly when anyone's printout is really finished.
This is a great boost to intra−office social relations.

The print queue software also spools the printouts on disk, i.e., the text is
kept in a file while the job is in the queue. This allows an application program
to spit out the print jobs quickly to the print queue software; the application
does not have to wait until the job is actually printed to continue. This is
really convenient, since it allows one to print out one version, and not have
to wait for it to be printed before one can make a completely revised new
version.

You can refer to the Printing−HOWTO located at

http://www.tldp.org/HOWTO/Printing−HOWTO/index.html for more help in
setting up printers.
2.3.11. The filesystem layout

The filesystem is divided into many parts; usually along the lines of a root
filesystem with /bin , /lib , /etc , /dev , and a few others; a /usr filesystem with
programs and unchanging data; /var filesystem with changing data (such as
log files); and a /home for everyone's personal files. Depending on the
hardware configuration and the decisions of the system administrator, the
division can be different; it can even be all in one filesystem.

Chapter 3 describes the filesystem layout in some little detail; the Filesystem
Hierarchy Standard . covers it in somewhat more detail. This can be found on
the web at: http://www.pathname.com/fhs/

Chapter 3. Overview of the Directory Tree

" Two days later, there was Pooh, sitting on his branch, dangling his legs, and
there, beside him, were four pots of honey..." (A.A. Milne)

This chapter describes the important parts of a standard Linux directory tree,
based on the Filesystem Hierarchy Standard . It outlines the normal way of
breaking the directory tree into separate filesystems with different purposes
and gives the motivation behind this particular split. Not all Linux
distributions follow this standard slavishly, but it is generic enough to give
you an overview.

3.1. Background

This chapter is loosely based on the Filesystems Hierarchy Standard (FHS).

version 2.1, which attempts to set a standard for how the directory tree in a
Linux system is organized. Such a standard has the advantage that it will be
easier to write or port software for Linux, and to administer Linux machines,
since everything should be in standardized places. There is no authority
behind the standard that forces anyone to comply with it, but it has gained
the support of many Linux distributions. It is not a good idea to break with
the FHS without very compelling reasons. The FHS attempts to follow Unix
tradition and current trends, making Linux systems familiar to those with
experience with other Unix systems, and vice versa.
This chapter is not as detailed as the FHS. A system administrator should
also read the full FHS for a complete understanding.

This chapter does not explain all files in detail. The intention is not to
describe every file, but to give an overview of the system from a filesystem
point of view. Further information on each file is available elsewhere in this
manual or in the Linux manual pages.

The full directory tree is intended to be breakable into smaller parts, each
capable of being on its own disk or partition, to accommodate to disk size
limits and to ease backup and other system administration tasks. The major
parts are the root (/ ), /usr , /var , and /home filesystems (see Figure 3−1).
Each part has a different purpose. The directory tree has been designed so
that it works well in a network of Linux machines which may share some
parts of the filesystems over a read−only device (e.g., a CD−ROM), or over
the network with NFS.

The root filesystem is specific for each machine (it is generally stored on a
local disk, although it could be a ramdisk or network drive as well) and
contains the files that are necessary for booting the system up, and to bring
it up to such a state that the other filesystems may be mounted. The
contents of the root filesystem will therefore be sufficient for the single user
state. It will also contain tools for fixing a broken system, and for recovering
lost files from backups.

The /usr filesystem contains all commands, libraries, manual pages, and
other unchanging files needed during normal operation. No files in /usr
should be specific for any given machine, nor should they be modified during
normal use. This allows the files to be shared over the network, which can be
cost−effective since it saves disk space (there can easily be hundreds of
megabytes, increasingly multiple gigabytes in /usr). It can make
administration easier (only the master /usr needs to be changed when
updating an application, not each machine separately) to have /usr network
mounted. Even if the filesystem is on a local disk, it could be mounted
read−only, to lessen the chance of filesystem corruption during a crash.

The /var filesystem contains files that change, such as spool directories (for
mail, news, printers, etc), log files, formatted manual pages, and temporary
files. Traditionally everything in /var has been somewhere below /usr , but
that made it impossible to mount /usr read−only.

The /home filesystem contains the users' home directories, i.e., all the real
data on the system. Separating home directories to their own directory tree
or filesystem makes backups easier; the other parts often do not have to be
backed up, or at least not as often as they seldom change. A big /home
might have to be broken across several filesystems, which requires adding
an extra naming level below /home, for example /home/students and
/home/staff.

Although the different parts have been called filesystems above, there is no
requirement that they actually be on separate filesystems. They could easily
be kept in a single one if the system is a small single−user system and the
user wants to keep things simple. The directory tree might also be divided
into filesystems differently, depending on how large the disks are, and how
space is allocated for various purposes. The important part, though, is that
all the standard names work; even if, say, /var and /usr are actually on the
same partition, the names /usr/lib/libc.a and /var/log/messages must work,
for example by moving files below /var into /usr/var, and making /var a
symlink to /usr/var.

The Unix filesystem structure groups files according to purpose, i.e., all
commands are in one place, all data files in another, documentation in a
third, and so on. An alternative would be to group files files according to the
program they belong to, i.e., all Emacs files would be in one directory, all TeX
in another, and so on. The problem with the latter approach is that it makes
it difficult to share files (the program directory often contains both static and
sharable and changing and non−sharable files), and sometimes to even find
the files (e.g., manual pages in a huge number of places, and making the
manual page programs find all of them is a maintenance nightmare).

3.2. The root filesystem

The root filesystem should generally be small, since it contains very critical
files and a small, infrequently modified filesystem has a better chance of not
getting corrupted. A corrupted root filesystem will generally mean that the
system becomes unbootable except with special measures (e.g., from a
floppy), so you don't want to risk it.

The root directory generally doesn't contain any files, except perhaps on
older systems where the standard boot image for the system, usually
called /vmlinuz was kept there. (Most distributions have moved those files
the the /boot directory. Otherwise, all files are kept in subdirectories under
the root filesystem:

/bin
Commands needed during bootup that might be used by normal users
(probably after bootup).

/sbin

Like /bin, but the commands are not intended for normal users, although
they may use them if necessary and allowed. /sbin is not usually in the
default path of normal users, but will be in root's default path.

/etc

Configuration files specific to the machine.

/root

The home directory for user root. This is usually not accessible to other users
on the system

/lib

Shared libraries needed by the programs on the root filesystem.

/lib/modules

Loadable kernel modules, especially those that are needed to boot the
system when recovering from disasters (e.g., network and filesystem
drivers).

/dev

Device files. These are special files that help the user interface with the
various devices on the system.

/tmp

Temporary files. As the name suggests, programs running often store

temporary files in here.

/boot

Files used by the bootstrap loader, e.g., LILO or GRUB. Kernel images are
often kept here instead of in the root directory. If there are many kernel
images, the directory can easily grow rather big, and it might be better to
keep it in a separate filesystem. Another reason would be to make sure the
kernel images are within the first 1024 cylinders of an IDE disk. This 1024
cylinder limit is no longer true in most cases. With modern BIOSes and later
versions of LILO (the LInux LOader) the 1024 cylinder limit can be passed
with logical block addressing (LBA). See the lilo manual page for more
details.

/mnt

Mount point for temporary mounts by the system administrator. Programs

aren't supposed to mount on /mnt automatically. /mnt might be divided into
subdirectories (e.g., /mnt/dosa might be the floppy drive using an MS−DOS
filesystem, and /mnt/exta might be the same with an ext2 filesystem).

/proc, /usr, /var, /home

Mount points for the other filesystems. Although /proc does not reside on any
disk in reality it is still mentioned here. See the section about /proc later in
the chapter.

3.3. The /etc directory

The /etc maintains a lot of files. Some of them are described below. For
others, you should determine which program they belong to and read the
manual page for that program. Many networking configuration files are in
/etc as well, and are described in the Networking Administrators' Guide.

/etc/rc or /etc/rc.d or /etc/rc?.d

Scripts or directories of scripts to run at startup or when changing the run

level. See Section 2.3.1 for further information.

/etc/passwd

The user database, with fields giving the username, real name, home
directory, and other information about each user. The format is documented
in the passwd manual page.

/etc/shadow

/etc/shadow is an encrypted file the holds user passwords.

/etc/fdprm

Floppy disk parameter table. Describes what different floppy disk formats
look like. Used by setfdprm . See the setfdprm manual page for more
information.

/etc/fstab

Lists the filesystems mounted automatically at startup by the mount −a

command (in /etc/rc or equivalent startup file). Under Linux, also contains
information about swap areas used automatically by swapon −a . See
Section 5.10.7 and the mount manual page for more information. Also fstab
usually has its own manual page in section 5.

/etc/group

Similar to /etc/passwd, but describes groups instead of users. See the group
manual page in section 5 for more information.

/etc/inittab

Configuration file for init.

/etc/issue

Output by getty before the login prompt. Usually contains a short description
or welcoming message to the system. The contents are up to the system
administrator.

/etc/magic

The configuration file for file. Contains the descriptions of various file formats
based on which file guesses the type of the file. See the magic and file
manual pages for more information.

/etc/motd

The message of the day, automatically output after a successful login.

Contents are up to the system administrator. Often used for getting
information to every user, such as warnings about planned downtimes.

/etc/mtab

List of currently mounted filesystems. Initially set up by the bootup scripts,

and updated automatically by the mount command. Used when a list of
mounted filesystems is needed, e.g., by the df command.

/etc/login.defs

Configuration file for the login command. The login.defs file usually has a
manual page in section 5.

/etc/printcap

Like /etc/termcap/etc/printcap , but intended for printers. However it uses

different syntax. The printcap has a manual page in section 5.

/etc/profile, /etc/bash.rc, /etc/csh.cshrc

Files executed at login or startup time by the Bourne, BASH , or C shells.
These allow the system administrator to set global defaults for all users.
Users can also create individual copies of these in their home directory to
personalize their environment. See the manual pages for the respective
shells.

/etc/securetty

Identifies secure terminals, i.e., the terminals from which root is allowed to
log in. Typically only the virtual consoles are listed, so that it becomes
impossible (or at least harder) to gain superuser privileges by breaking into a
system over a modem or a network. Do not allow root logins over a network.
Prefer to log in as an unprivileged user and use su or sudo to gain root
privileges.

/etc/shells

Lists trusted shells. The chsh command allows users to change their login
shell only to shells listed in this file. ftpd, is the server process that provides
FTP services for a machine, will check that the user's shell is listed in
/etc/shells and will not let people log in unless the shell is listed there.

/etc/termcap

The terminal capability database. Describes by what ``escape sequences''

various terminals can be controlled. Programs are written so that instead of
directly outputting an escape sequence that only works on a particular brand
of terminal, they look up the correct sequence to do whatever it is they want
to do in /etc/termcap. As a result most programs work with most kinds of
terminals. See the termcap, curs_termcap, and terminfo manual pages for
more information.

3.4. The /dev directory

The /dev directory contains the special device files for all the devices. The
device files are created during installation, and later with the /dev/MAKEDEV
script. The /dev/MAKEDEV.local is a script written by the system
administrator that creates local−only device files or links (i.e. those that are
not part of the standard MAKEDEV, such as device files for some
non−standard device driver).

This list which follows is by no means exhaustive or as detailed as it could

be. Many of these device files will need support compiled into your kernel for
the hardware. Read the kernel documentation to find details of any particular
device.

If you think there are other devices which should be included here but aren't
then let me know. I will try to include them in the next revision.

/dev/dsp

Digital Signal Processor. Basically this forms the interface between software
which produces sound and your soundcard. It is a character device on major
node 14 and minor 3.

/dev/fd0

The first floppy drive. If you are lucky enough to have several drives then
they will be numbered sequentially. It is a character device on major node 2
and minor 0.

/dev/fb0

The first framebuffer device. A framebuffer is an abstraction layer between

software and graphics hardware. This means that applications do not need to
know about what kind of hardware you have but merely how to communicate
with the framebuffer driver's API (Application Programming Interface) which
is well defined and standardized. The framebuffer is a character device and
is on major node 29 and minor 0.

/dev/hda

/dev/hda is the master IDE drive on the primary IDE controller. /dev/hdb the
slave drive on the primary controller. /dev/hdc , and /dev/hdd are the master
and slave devices on the secondary controller respectively. Each disk is
divided into partitions. Partitions 1−4 are primary partitions and partitions 5
and above are logical partitions inside extended partitions. Therefore the
device file which references each partition is made up of several parts. For
example /dev/hdc9 references partition 9 (a logical partition inside an
extended partition type) on the master IDE drive on the secondary IDE
controller. The major and minor node numbers are somewhat complex. For
the first IDE controller all partitions are block devices on major node 3. The
master drive hda is at minor 0 and the slave drive hdb is at minor 64. For
each partition inside the drive add the partition number to the minor minor
node number for the drive. For example /dev/hdb5 is major 3, minor 69 (64 +
5 = 69). Drives on the secondary interface are handled the same way, but
with major node 22.
/dev/ht0

The first IDE tape drive. Subsequent drives are numbered ht1 etc. They are
character devices on major node 37 and start at minor node 0 for ht0 1 for
ht1 etc.

/dev/js0

The first analogue joystick. Subsequent joysticks are numbered js1, js2 etc.
Digital joysticks are called djs0, djs1 and so on. They are character devices
on major node 15. The analogue joysticks start at minor node 0 and go up to
127 (more than enough for even the most fanatic gamer). Digital joysticks
start at minor node 128.

/dev/lp0

The first parallel printer device. Subsequent printers are numbered lp1, lp2
etc. They are character devices on major mode 6 and minor nodes starting at
0 and numbered sequentially.

/dev/loop0

The first loopback device. Loopback devices are used for mounting
filesystems which are not located on other block devices such as disks. For
example if you wish to mount an iso9660 CD ROM image without burning it
to CD then you need to use a loopback device to do so. This is usually
transparent to the user and is handled by the mount command. Refer to the
manual pages for mount and losetup. The loopback devices are block
devices on major node 7 and with minor nodes starting at 0 and numbered
sequentially.

/dev/md0

First metadisk group. Metadisks are related to RAID (Redundant Array of

Independent Disks) devices. Please refer to the most current RAID HOWTO at
the LDP for more details. This can be found at
http://www.tldp.org/HOWTO/Software−RAID−HOWTO.html. Metadisk devices
are block devices on major node 9 with minor nodes starting at 0 and
numbered sequentially.

/dev/mixer

This is part of the OSS (Open Sound System) driver. Refer to the OSS
documentation at http://www.opensound.com for more details. It is a
character device on major node 14, minor node 0.
/dev/null

The bit bucket. A black hole where you can send data for it never to be seen
again. Anything sent to /dev/null will disappear. This can be useful if, for
example, you wish to run a command but not have any feedback appear on
the terminal. It is a character device on major node 1 and minor node 3.

/dev/psaux

The PS/2 mouse port. This is a character device on major node 10, minor
node 1.

/dev/pda

Parallel port IDE disks. These are named similarly to disks on the internal IDE
controllers

(/dev/hd*). They are block devices on major node 45. Minor nodes need
slightly more explanation here. The first device is /dev/pda and it is on minor
node 0. Partitions on this device are found by adding the partition number to
the minor number for the device. Each device is limited to 15 partitions each
rather than 63 (the limit for internal IDE disks). /dev/pdb minor nodes start at
16, /dev/pdc at 32 and /dev/pdd at 48. So for example the minor node
number for /dev/pdc6 would be 38 (32 + 6 = 38). This scheme limits you to 4
parallel disks of 15 partitions each.

/dev/pcd0

Parallel port CD ROM drives. These are numbered from 0 onwards. All are
block devices on major node 46. /dev/pcd0 is on minor node 0 with
subsequent drives being on minor nodes 1, 2, 3 etc.

/dev/pt0

Parallel port tape devices. Tapes do not have partitions so these are just
numbered sequentially. They are character devices on major node 96. The
minor node numbers start from 0 for /dev/pt0, 1 for /dev/pt1, and so on.

/dev/parport0

The raw parallel ports. Most devices which are attached to parallel ports
have their own drivers. This is a device to access the port directly. It is a
character device on major node 99 with minor node 0. Subsequent devices
after the first are numbered sequentially incrementing the minor node.

/dev/random or /dev/urandom
These are kernel random number generators. /dev/random is a
non−deterministic generator which means that the value of the next number
cannot be guessed from the preceding ones. It uses the entropy of the
system hardware to generate numbers. When it has no more entropy to use
then it must wait until it has collected more before it will allow any more
numbers to be read from it.

/dev/urandom works similarly. Initially it also uses the entropy of the system
hardware, but when there is no more entropy to use it will continue to return
numbers using a pseudo random number generating formula. This is
considered to be less secure for vital purposes such as cryptographic key
pair generation. If security is your overriding concern then use /dev/random,
if speed is more important then /dev/urandom works fine. They are character
devices on major node 1 with minor nodes 8 for /dev/random and 9 for
/dev/urandom.

/dev/sda

The first SCSI drive on the first SCSI bus. The following drives are named
similar to IDE drives. /dev/sdb is the second SCSI drive, /dev/sdc is the third
SCSI drive, and so forth.

/dev/ttyS0

The first serial port. Many times this it the port used to connect an external
modem to your system.

/dev/zero

This is a simple way of getting many 0s. Every time you read from this
device it will return 0. This can be useful sometimes, for example when you
want a file of fixed length but don't really care what it contains. It is a
character device on major node 1 and minor node 5.

3.5. The /usr filesystem.

The /usr filesystem is often large, since all programs are installed there. All
files in /usr usually come from a Linux distribution; locally installed programs
and other stuff goes below /usr/local. This makes it possible to update the
system from a new version of the distribution, or even a completely new
distribution, without having to install all programs again. Some of the
subdirectories of /usr are listed below (some of the less important directories
have been dropped; see the FSSTND for more information).

/usr/X11R6.
The X Window System, all files. To simplify the development and installation
of X, the X files have not been integrated into the rest of the system. There is
a directory tree below /usr/X11R6 similar to that below /usr itself.

/usr/bin.

Almost all user commands. Some commands are in /bin or in /usr/local/bin.

/usr/sbin

System administration commands that are not needed on the root

filesystem, e.g., most server programs.

/usr/share/man, /usr/share/info, /usr/share/doc

Manual pages, GNU Info documents, and miscellaneous other documentation

files, respectively.

/usr/include

Header files for the C programming language. This should actually be

below /usr/lib for consistency, but the tradition is overwhelmingly in support
for this name.

/usr/lib

Unchanging data files for programs and subsystems, including some

site−wide configuration files. The name lib comes from library; originally
libraries of programming subroutines were stored in

/usr/lib.

/usr/local

The place for locally installed software and other files. Distributions may not
install anything in here. It is reserved solely for the use of the local
administrator. This way he can be absolutely certain that no updates or
upgrades to his distribution will overwrite any extra software he has installed
locally.

3.6. The /var filesystem

The /var contains data that is changed when the system is running normally.
It is specific for each system, i.e. not shared over the network with other
computers.

/var/cache/man
A cache for man pages that are formatted on demand. The source for manual
pages is usually stored in /usr/share/man/man?/ (where ? is the manual
section. See the manual page for man in section 7); some manual pages
might come with a pre−formatted version, which might be stored in
/usr/share/man/cat* . Other manual pages need to be formatted when they
are first viewed; the formatted version is then stored in /var/cache/man so
that the next person to view the same page won't have to wait for it to be
formatted.

/var/games

Any variable data belonging to games in /usr should be placed here. This is in
case /usr is mounted read only.

/var/lib

Files that change while the system is running normally.

/var/local

Variable data for programs that are installed in /usr/local (i.e., programs that
have been installed by the system administrator). Note that even locally
installed programs should use the other /var directories if they are
appropriate, e.g., /var/lock.

/var/lock

Lock files. Many programs follow a convention to create a lock file in /var/lock
to indicate that they are using a particular device or file. Other programs will
notice the lock file and won't attempt to use the device or file.

/var/log

Log files from various programs, especially login(/var/log/wtmp, which logs

all logins and logouts into the system) and syslog(/var/log/messages, where
all kernel and system program message are usually stored). Files in /var/log
can often grow indefinitely, and may require cleaning at regular intervals.

/var/mail

This is the FHS approved location for user mailbox files. Depending on how
far your distribution has gone towards FHS compliance, these files may still
be held in /var/spool/mail.

/var/run
Files that contain information about the system that is valid until the system
is next booted. For example, /var/run/utmp contains information about
people currently logged in.

/var/spool

Directories for news, printer queues, and other queued work. Each different
spool has its own subdirectory below /var/spool, e.g., the news spool is in
/var/spool/news . Note that some installations which are not fully compliant
with the latest version of the FHS may have user mailboxes under
/var/spool/mail.

/var/tmp

Temporary files that are large or that need to exist for a longer time than
what is allowed for /tmp .

(Although the system administrator might not allow very old files in /var/tmp
either.)

3.7. The /proc filesystem

The /proc filesystem contains a illusionary filesystem. It does not exist on a

disk. Instead, the kernel creates it in memory. It is used to provide
information about the system (originally about processes, hence the name).
Some of the more important files and directories are explained below. The
/proc filesystem is described in more detail in the proc manual page.

/proc/1

A directory with information about process number 1. Each process has a

directory below /proc with the name being its process identification number.

/proc/cpuinfo

Information about the processor, such as its type, make, model, and
performance.

/proc/devices

List of device drivers configured into the currently running kernel.

/proc/dma

Shows which DMA channels are being used at the moment.

/proc/filesystems
Filesystems configured into the kernel.

/proc/interrupts

Shows which interrupts are in use, and how many of each there have been.

/proc/ioports

Which I/O ports are in use at the moment.

/proc/kcore

An image of the physical memory of the system. This is exactly the same
size as your physical memory, but does not really take up that much
memory; it is generated on the fly as programs access it. (Remember: unless
you copy it elsewhere, nothing under /proc takes up any disk space at all.)

/proc/kmsg

Messages output by the kernel. These are also routed to syslog.

/proc/ksyms

Symbol table for the kernel.

/proc/loadavg

The `load average' of the system; three meaningless indicators of how much
work the system has to do at the moment.

/proc/meminfo

Information about memory usage, both physical and swap.

/proc/modules

Which kernel modules are loaded at the moment.

/proc/net

Status information about network protocols.

/proc/self

A symbolic link to the process directory of the program that is looking at

/proc. When two processes look at /proc, they get different links. This is
mainly a convenience to make it easier for programs to get at their process
directory.

/proc/stat
Various statistics about the system, such as the number of page faults since
the system was booted.

/proc/uptime

The time the system has been up.

/proc/version

The kernel version.

Note that while the above files tend to be easily readable text files, they can
sometimes be formatted in a way that is not easily digestible. There are
many commands that do little more than read the above files and format
them for easier understanding. For example, the freeprogram reads
/proc/meminfo converts the amounts given in bytes to kilobytes (and adds a
little more information, as well).

Chapter 4. Hardware, Devices, and Tools

"Knowledge speaks, but wisdom listens." Jimi Hendrix

This chapter gives an overview of what a device file is, and how to create
one. The canonical list of device files is
/usr/src/linux/Documentation/devices.txt if you have the Linux kernel source
code installed on your system. The devices listed here are correct as of
kernel version 2.6.8.

4.1. Hardware Utilities

4.1.1. The MAKEDEV Script

Most device files will already be created and will be there ready to use after
you install your Linux system. If by some chance you need to create one
which is not provided then you should first try to use the MAKEDEV script.
This script is usually located in /dev/MAKEDEV but might also have a copy (or
a symbolic link) in /sbin/MAKEDEV. If it turns out not to be in your path then
you will need to specify the path to it explicitly.

In general the command is used as: # /dev/MAKEDEV −v ttyS0 create ttyS0

c 4 64 root:dialout 0660

This will create the device file /dev/ttyS0 with major node 4 and minor node
64 as a character device with access permissions 0660 with owner root and
group dialout. ttyS0 is a serial port. The major and minor node numbers are
numbers understood by the kernel. The kernel refers to hardware devices as
numbers, this would be very difficult for us to remember, so we use
filenames. Access permissions of 0660 means read and write permission for
the owner (root in this case) and read and write permission for members of
the group (dialout in this case) with no access for anyone else.

4.1.2. The mknod command

MAKEDEV is the preferred way of creating device files which are not present.
However sometimes the MAKEDEV script will not know about the device file
you wish to create. This is where the mknod command comes in. In order to
use mknod you need to know the major and minor node numbers for the
device you wish to create. The devices.txt file in the kernel source
documentation is the canonical source of this information.

To take an example, let us suppose that our version of the MAKEDEV script
does not know how to create the

/dev/ttyS0 device file. We need to use mknod to create it. We know from
looking at the devices.txt that it should be a character device with major
number 4 and minor number 64. So we now know all we need to create the
file.

# mknod /dev/ttyS0 c 4 64

# chown root.dialout /dev/ttyS0

# chmod 0644 /dev/ttyS0

# ls −l /dev/ttyS0 crw−rw−−−− 1 root dialout 4, 64 Oct 23 18:23

/dev/ttyS

As you can see, many more steps are required to create the file. In this
example you can see the process required however. It is unlikely in the
extreme that the ttyS0 file would not be provided by the MAKEDEV script, but
it suffices to illustrate the point.

4.1.3. The lspci command

lspci

To be added

4.1.4. The lsdev command

lsdev

To be added
4.1.5. The lsusb command

lsusb

To be added

4.1.6. The lsraid command

lsraid

To be added

4.1.7. The hdparm command

hdparm

To be added

4.1.8. More Hardware Resources

More information on what hardware resources the kernel is using can be

found in the /proc directory. Refer to Section 3.7 in chapter 3.

4.2. Kernel Modules

This section will discuss kernel modules.

To be added

4.2.1. lsmod

lsmod

To be added

4.2.2. insmod

insmod

To be added

4.2.3. depmod

depmod

To be added

4.2.4. rmmod

rmmod

To be added
4.2.5. modprobe

Modprobe-

To be added

Chapter 5. Using Disks and Other Storage Media

"On a clear disk you can seek forever. When you install or upgrade your
system, you need to do a fair amount of work on your disks. You have to
make filesystems on your disks so that files can be stored on them and
reserve space for the different parts of your system.

This chapter explains all these initial activities. Usually, once you get your
system set up, you won't have to go through the work again, except for using
floppies. You'll need to come back to this chapter if you add a new disk or
want to fine−tune your disk usage.

The basic tasks in administering disks are:

Format your disk. This does various things to prepare it for use, such as
checking for bad sectors. (Formatting is nowadays not necessary for most
hard disks.)

Partition a hard disk, if you want to use it for several activities that aren't
supposed to interfere with one another. One reason for partitioning is to store
different operating systems on the same disk. Another reason is to keep user
files separate from system files, which simplifies back−ups and helps protect
the system files from corruption.

Make a filesystem (of a suitable type) on each disk or partition. The disk
means nothing to Linux until you make a filesystem; then files can be
created and accessed on it.

Mount different filesystems to form a single tree structure, either

automatically, or manually as needed. (Manually mounted filesystems
usually need to be unmounted manually as well.) Chapter 6 contains
information about virtual memory and disk caching, of which you also need
to be aware when using disks.

5.1. Two kinds of devices

UNIX, and therefore Linux, recognizes two different kinds of device:

random−access block devices (such as disks), and character devices (such
as tapes and serial lines) , some of which may be serial, and some
random−access. Each supported device is represented in the filesystem as a
device file. When you read or write a device file, the data comes from or
goes to the device it represents. This way no special programs (and no
special application programming methodology, such as catching interrupts or
polling a serial port) are necessary to access devices; for example, to send a
file to the printer, one could just say cat filename > /dev/lp1 and the
contents of the file are printed (the file must, of course, be in a form that the
printer understands). However, since it is not a good idea to have several
people cat their files to the printer at the same time, one usually uses a
special program to send the files to be printed (usually lpr ). This program
makes sure that only one file is being printed at a time, and will
automatically send files to the printer as soon as it finishes with the previous
file. Something similar is needed for most devices. In fact, one seldom needs
to worry about device files at all.

Since devices show up as files in the filesystem (in the /dev directory), it is
easy to see just what device files exist, using ls or another suitable
command. In the output of ls −l, the first column contains the type of the file
and its permission .

The first character in the first column, i.e., `c' in crw−rw−rw− above, tells an
informed user the type of the file, in this case a character device. For
ordinary files, the first character is `−', for directories it is `d', and for block
devices `b'; see the ls man page for further information.

Note that usually all device files exist even though the device itself might be
not be installed. So just because you have a file /dev/sda, it doesn't mean
that you really do have an SCSI hard disk. Having all the device files makes
the installation programs simpler, and makes it easier to add new hardware
(there is no need to find out the correct parameters for and create the device
files for the new device).

5.2. Hard disks

This subsection introduces terminology related to hard disks. If you already

know the terms and concepts, you can skip this subsection.

A hard disk consists of one or more circular aluminum platters\ , of which

either or both surfaces are coated with a magnetic substance used for
recording the data. For each surface, there is a read−write head that
examines or alters the recorded data. The platters rotate on a common axis;
typical rotation speed is 5400 or 7200 rotations per minute, although
high−performance hard disks have higher speeds and older disks may have
lower speeds. The heads move along the radius of the platters; this
movement combined with the rotation of the platters allows the head to
access all parts of the surfaces.

The processor (CPU) and the actual disk communicate through a disk
controller . This relieves the rest of the computer from knowing how to use
the drive, since the controllers for different types of disks can be made to
use the same interface towards the rest of the computer. Therefore, the
computer can say just ``hey disk, give me what I want'', instead of a long
and complex series of electric signals to move the head to the proper
location and waiting for the correct position to come under the head and
doing all the other unpleasant stuff necessary. (In reality, the interface to the
controller is still complex, but much less so than it would otherwise be.) The
controller may also do other things, such as caching, or automatic bad sector
replacement.

The above is usually all one needs to understand about the hardware. There
are also other things, such as the motor that rotates the platters and moves
the heads, and the electronics that control the operation of the mechanical
parts, but they are mostly not relevant for understanding the working
principles of a hard disk.

The surfaces are usually divided into concentric rings, called tracks, and
these in turn are divided into sectors. This division is used to specify
locations on the hard disk and to allocate disk space to files. To find a given
place on the hard disk, one might say ``surface 3, track 5, sector 7''. Usually
the number of sectors is the same for all tracks, but some hard disks put
more sectors in outer tracks (all sectors are of the same physical size, so
more of them fit in the longer outer tracks). Typically, a sector will hold 512
bytes of data. The disk itself can't handle smaller amounts of data than one
sector.

Each surface is divided into tracks (and sectors) in the same way. This means
that when the head for one surface is on a track, the heads for the other
surfaces are also on the corresponding tracks. All the corresponding tracks
taken together are called a cylinder. It takes time to move the heads from
one track (cylinder) to another, so by placing the data that is often accessed
together (say, a file) so that it is within one cylinder, it is not necessary to
move the heads to read all of it. This improves performance. It is not always
possible to place files like this; files that are stored in several places on the
disk are called fragmented.
The number of surfaces (or heads, which is the same thing), cylinders, and
sectors vary a lot; the specification of the number of each is called the
geometry of a hard disk. The geometry is usually stored in a special,
battery−powered memory location called the CMOS RAM , from where the
operating system can fetch it during bootup or driver initialization.

Unfortunately, the BIOS has a design limitation, which makes it impossible to

specify a track number that is larger than 1024 in the CMOS RAM, which is
too little for a large hard disk. To overcome this, the hard disk controller lies
about the geometry, and translates the addresses given by the computer
into something that fits reality. For example, a hard disk might have 8 heads,
2048 tracks, and 35 sectors per track. Its controller could lie to the computer
and claim that it has 16 heads, 1024 tracks, and 35 sectors per track, thus
not exceeding the limit on tracks, and translates the address that the
computer gives it by halving the head number, and doubling the track
number. The mathematics can be more complicated in reality, because the
numbers are not as nice as here (but again, the details are not relevant for
understanding the principle). This translation distorts the operating system's
view of how the disk is organized, thus making it impractical to use the
all−data−on−one−cylinder trick to boost performance.

The translation is only a problem for IDE disks. SCSI disks use a sequential
sector number (i.e., the controller translates a sequential sector number to a
head, cylinder, and sector triplet), and a completely different method for the
CPU to talk with the controller, so they are insulated from the problem. Note,
however, that the computer might not know the real geometry of an SCSI
disk either.

Since Linux often will not know the real geometry of a disk, its filesystems
don't even try to keep files within a single cylinder. Instead, it tries to assign
sequentially numbered sectors to files, which almost always gives similar
performance. The issue is further complicated by on−controller caches, and
automatic prefetches done by the controller.

Each hard disk is represented by a separate device file. There can (usually)
be only two or four IDE hard disks. These are known as /dev/hda, /dev/hdb,
/dev/hdc, and /dev/hdd, respectively. SCSI hard disks are known as
/dev/sda, /dev/sdb, and so on. Similar naming conventions exist for other
hard disk types; see Chapter 4 for more information. Note that the device
files for the hard disks give access to the entire disk, with no regard to
partitions (which will be discussed below), and it's easy to mess up the
partitions or the data in them if you aren't careful. The disks' device files are
usually used only to get access to the master boot record (which will also be
discussed below).

5.3. Storage Area Networks − Draft

A SAN is a dedicated storage network that provides block level access to

LUNs. A LUN, or logical unit number, is a virtual disk provided by the SAN.
The system administrator the same access and rights to the LUN as if it were
a disk directly attached to it. The administrator can partition, and format the
disk in any means he or she chooses.

Two networking protocols commonly used in a SAN are fibre channel and
iSCSI . A fibre channel network is very fast and is not burdened by the other
network traffic in a company's LAN. However, it's very expensive. Fibre
channel cards cost around $1000.00 USD each. They also require special
fibre channel switches.

iSCSI is a newer technology that sends SCSI commands over a TCP/IP

network. While this method may not be as fast as a Fibre Channel network, it
does save money by using less expensive network hardware.More To Be
Added

5.4. Network Attached Storage − Draft

A NAS uses your companies existing Ethernet network to allow access to

shared disks. This is filesystem level access. The system administrator does
not have the ability to partition or format the disks since they are potentially
shared by multiple computers. This technology is commonly used to provide
multiple workstations access to the same data.

Similar to a SAN, a NAS need to make use of a protocol to allow access to it's
disks. With a NAS this is either CIFS/Samba , or NFS.

Traditionally CIFS was used with Microsoft Windows networks, and NFS was
used with UNIX & Linux networks. However, with Samba, Linux machines can
also make use of CIFS shares.

Does this mean that your Windows 2003 server or your Linux box are NAS
servers because they provide access to shared drives over your network?
Yes, they are. You could also purchase a NAS device from a number of
manufacturers. These devices are specifically designed to provide high speed
access to data. More To Be Added

5.4.1. NFS
To be added

5.4.2. CIFS

To be added

5.5. Floppies

A floppy disk consists of a flexible membrane covered on one or both sides

with similar magnetic substance as a hard disk. The floppy disk itself doesn't
have a read−write head, that is included in the drive. A floppy corresponds
to one platter in a hard disk, but is removable and one drive can be used to
access different floppies, and the same floppy can be read by many drives,
whereas the hard disk is one indivisible unit.

Like a hard disk, a floppy is divided into tracks and sectors (and the two
corresponding tracks on either side of a floppy form a cylinder), but there are
many fewer of them than on a hard disk.

A floppy drive can usually use several different types of disks; for example, a
3.5 inch drive can use both 720 KB and 1.44 MB disks. Since the drive has to
operate a bit differently and the operating system must know how big the
disk is, there are many device files for floppy drives, one per combination of
drive and disk type. Therefore, /dev/fd0H1440 is the first floppy drive (fd0),
which must be a 3.5 inch drive, using a 3.5 inch, high density disk (H) of size
1440 KB (1440), i.e., a normal 3.5 inch HD floppy.

The names for floppy drives are complex, however, and Linux therefore has a
special floppy device type that automatically detects the type of the disk in
the drive. It works by trying to read the first sector of a newly inserted floppy
using different floppy types until it finds the correct one. This naturally
requires that the floppy is formatted first. The automatic devices are called
/dev/fd0, /dev/fd1, and so on.

The parameters the automatic device uses to access a disk can also be set
using the program setfdprm . This can be useful if you need to use disks that
do not follow any usual floppy sizes, e.g., if they have an unusual number of
sectors, or if the autodetecting for some reason fails and the proper device
file is missing.

Linux can handle many nonstandard floppy disk formats in addition to all the
standard ones. Some of these require using special formatting programs.
We'll skip these disk types for now, but in the mean time you can examine
the /etc/fdprm file. It specifies the settings that setfdprm recognizes.
The operating system must know when a disk has been changed in a floppy
drive, for example, in order to avoid using cached data from the previous
disk. Unfortunately, the signal line that is used for this is sometimes broken,
and worse, this won't always be noticeable when using the drive from within
MS−DOS. If you are experiencing weird problems using floppies, this might
be the reason. The only way to correct it is to repair the floppy drive.

5.6. CD−ROMs

A CD−ROM drive uses an optically read, plastic coated disk. The information
is recorded on the surface of the disk in small `holes' aligned along a spiral
from the center to the edge. The drive directs a laser beam along the spiral
to read the disk. When the laser hits a hole, the laser is reflected in one way;
when it hits smooth surface, it is reflected in another way. This makes it easy
to code bits, and therefore information. The rest is easy, mere mechanics.

CD−ROM drives are slow compared to hard disks. Whereas a typical hard
disk will have an average seek time less than 15 milliseconds, a fast
CD−ROM drive can use tenths of a second for seeks. The actual data transfer
rate is fairly high at hundreds of kilobytes per second. The slowness means
that CD−ROM drives are not as pleasant to use as hard disks (some Linux
distributions provide `live' filesystems on CD−ROMs, making it unnecessary
to copy the files to the hard disk, making installation easier and saving a lot
of hard disk space), although it is still possible. For installing new software,
CD−ROMs are very good, since maximum speed is not essential during
installation.

There are several ways to arrange data on a CD−ROM. The most popular one
is specified by the international standard ISO 9660 . This standard specifies a
very minimal filesystem, which is even more crude than the one MS−DOS
uses. On the other hand, it is so minimal that every operating system should
be able to map it to its native system.

For normal UNIX use, the ISO 9660 filesystem is not usable, so an extension
to the standard has been developed, called the Rock Ridge extension. Rock
Ridge allows longer filenames, symbolic links, and a lot of other goodies,
making a CD−ROM look more or less like any contemporary UNIX filesystem.
Even better, a Rock Ridge filesystem is still a valid ISO 9660 filesystem,
making it usable by non−UNIX systems as well. Linux supports both ISO
9660 and the Rock Ridge extensions; the extensions are recognized and used
automatically.
The filesystem is only half the battle, however. Most CD−ROMs contain data
that requires a special program to access, and most of these programs do
not run under Linux (except, possibly, under dosemu, the Linux MS−DOS
emulator, or wine, the Windows emulator.

Ironically perhaps, wine actually stands for ``Wine Is Not an Emulator''. Wine,
more strictly, is an API (Application Program Interface) replacement. Please
see the wine documentation at http://www.winehq.com for more information.

There is also VMWare, a commercial product, which emulates an entire x86

machine in software. See the VMWare website, http://www.vmware.com for
more information.

A CD−ROM drive is accessed via the corresponding device file. There are
several ways to connect a CD−ROM drive to the computer: via SCSI, via a
sound card, or via EIDE. The hardware hacking needed to do this is outside
the scope of this book, but the type of connection decides the device file.

5.7. Tapes

A tape drive uses a tape, similar to cassettes used for music. A tape is serial
in nature, which means that in order to get to any given part of it, you first
have to go through all the parts in between. A disk can be accessed
randomly, i.e., you can jump directly to any place on the disk. The serial
access of tapes makes them slow.

On the other hand, tapes are relatively cheap to make, since they do not
need to be fast. They can also easily be made quite long, and can therefore
contain a large amount of data. This makes tapes very suitable for things like
archiving and backups, which do not require large speeds, but benefit from
low costs and large storage capacities.

5.8. Formatting

Formatting is the process of writing marks on the magnetic media that are
used to mark tracks and sectors. Before a disk is formatted, its magnetic
surface is a complete mess of magnetic signals. When it is formatted, some
order is brought into the chaos by essentially drawing lines where the tracks
go, and where they are divided into sectors. The actual details are not quite
exactly like this, but that is irrelevant. What is important is that a disk cannot
be used unless it has been formatted.

The terminology is a bit confusing here: in MS−DOS and MS Windows, the

word formatting is used to cover also the process of creating a filesystem
(which will be discussed below). There, the two processes are often
combined, especially for floppies. When the distinction needs to be made,
the real formatting is called low−level formatting, while making the
filesystem is called high−level formatting . In UNIX circles, the two are called
formatting and making a filesystem, so that's what is used in this book as
well.

For IDE and some SCSI disks the formatting is actually done at the factory
and doesn't need to be repeated; hence most people rarely need to worry
about it. In fact, formatting a hard disk can cause it to work less well, for
example because a disk might need to be formatted in some very special
way to allow automatic bad sector replacement to work.

Disks that need to be or can be formatted often require a special program

anyway, because the interface to the formatting logic inside the drive is
different from drive to drive. The formatting program is often either on the
controller BIOS, or is supplied as an MS−DOS program; neither of these can
easily be used from within Linux.

During formatting one might encounter bad spots on the disk, called bad
blocks or bad sectors. These are sometimes handled by the drive itself, but
even then, if more of them develop, something needs to be done to avoid
using those parts of the disk. The logic to do this is built into the filesystem;
how to add the information into the filesystem is described below.
Alternatively, one might create a small partition that covers just the bad part
of the disk; this approach might be a good idea if the bad spot is very large,
since filesystems can sometimes have trouble with very large bad areas.

Floppies are formatted with fdformat . The floppy device file to use is given
as the parameter. For example, the following command would format a high
density, 3.5 inch floppy in the first floppy drrive.

Note that if you want to use an autodetecting device (e.g., /dev/fd0), you
must set the parameters of the device with setfdprm first. It is usually more
convenient to choose the correct device file that matches the type of the
floppy. Note that it is unwise to format floppies to contain more information
than what they are designed for. fdformatalso validate the floppy, i.e., check
it for bad blocks. It will try a bad block several times (you can usually hear
this, the drive noise changes dramatically). If the floppy is only marginally
bad (due to dirt on the read/write head, some errors are false signals),
fdformat won't complain, but a real error will abort the validation process.
The kernel will print log messages for each I/O error it finds; these will go to
the console or, if syslog is being used, to the file /var/log/messages. fdformat
itself won't tell where the error is (one usually doesn't care, floppies are
cheap enough that a bad one is automatically thrown away).

The badblocks command can be used to search any disk or partition for bad
blocks (including a floppy). It does not format the disk, so it can be used to
check even existing filesystems. The example below checks a 3.5 inch floppy
with two bad blocks.

badblocks outputs the block numbers of the bad blocks it finds. Most
filesystems can avoid such bad blocks. They maintain a list of known bad
blocks, which is initialized when the filesystem is made, and can be modified
later. The initial search for bad blocks can be done by the mkfs command
(which initializes the filesystem), but later checks should be done with
badblocks and the new blocks should be added with fsck. We'll describe mkfs
and fsck later.

Many modern disks automatically notice bad blocks, and attempt to fix them
by using a special, reserved good block instead. This is invisible to the
operating system. This feature should be documented in the disk's manual, if
you're curious if it is happening. Even such disks can fail, if the number of
bad blocks grows too large, although chances are that by then the disk will
be so rotten as to be unusable.

5.9. Partitions

A hard disk can be divided into several partitions. Each partition functions as
if it were a separate hard disk.

The idea is that if you have one hard disk, and want to have, say, two
operating systems on it, you can divide the disk into two partitions. Each
operating system uses its partition as it wishes and doesn't touch the other
ones. This way the two operating systems can co−exist peacefully on the
same hard disk. Without partitions one would have to buy a hard disk for
each operating system.

Floppies are not usually partitioned. There is no technical reason against this,
but since they're so small, partitions would be useful only very rarely.
CD−ROMs are usually also not partitioned, since it's easier to use them as
one big disk, and there is seldom a need to have several operating systems
on one.

5.9.1. The MBR, boot sectors and partition table

The information about how a hard disk has been partitioned is stored in its
first sector (that is, the first sector of the first track on the first disk surface).
The first sector is the master boot record (MBR) of the disk; this is the sector
that the BIOS reads in and starts when the machine is first booted. The
master boot record contains a small program that reads the partition table,
checks which partition is active (that is, marked bootable), and reads the first
sector of that partition, the partition's boot sector (the MBR is also a boot
sector, but it has a special status and therefore a special name). This boot
sector contains another small program that reads the first part of the
operating system stored on that partition (assuming it is bootable), and then
starts it.

The partitioning scheme is not built into the hardware, or even into the BIOS.
It is only a convention that many operating systems follow. Not all operating
systems do follow it, but they are the exceptions. Some operating systems
support partitions, but they occupy one partition on the hard disk, and use
their internal partitioning method within that partition. The latter type exists
peacefully with other operating systems (including Linux), and does not
require any special measures, but an operating system that doesn't support
partitions cannot co−exist on the same disk with any other operating
system.

As a safety precaution, it is a good idea to write down the partition table on a

piece of paper, so that if it ever corrupts you don't have to lose all your files.
(A bad partition table can be fixed with fdisk). The relevant information is
given by the fdisk −l command.

5.9.2. Extended and logical partitions

The original partitioning scheme for PC hard disks allowed only four
partitions. This quickly turned out to be too little in real life, partly because
some people want more than four operating systems (Linux, MS−DOS, OS/2,
Minix, FreeBSD, NetBSD, or Windows/NT, to name a few), but primarily
because sometimes it is a good idea to have several partitions for one
operating system. For example, swap space is usually best put in its own
partition for Linux instead of in the main Linux partition for reasons of speed
(see below).

To overcome this design problem, extended partitions were invented. This

trick allows partitioning a primary partition into sub−partitions. The primary
partition thus subdivided is the extended partition; the sub−partitions are
logical partitions. They behave like primary partitions, but are created
differently. There is no speed difference between them. By using an extended
partition you can now have up to 15 partitions per disk.

The disk is divided into three primary partitions, the second of which is
divided into two logical partitions. Part of the disk is not partitioned at all.
The disk as a whole and each primary partition has a boot sector.

5.9.3. partition types

The partition tables (the one in the MBR, and the ones for extended
partitions) contain one byte per partition that identifies the type of that
partition. This attempts to identify the operating system that uses the
partition, or what it uses it for. The purpose is to make it possible to avoid
having two operating systems accidentally using the same partition.
However, in reality, operating systems do not really care about the partition
type byte; e.g., Linux doesn't care at all what it is. Worse, some of them use
it incorrectly; e.g., at least some versions of DR−DOS ignore the most
significant bit of the byte, while others don't.

There is no standardization agency to specify what each byte value means,

but as far as Linux is concerned, here is a list of partition types as per the
fdisk program.

Note that even though it is called a filesystem, no part of the proc filesystem
touches any disk. It exists only in the kernel's imagination. Whenever anyone
tries to look at any part of the proc filesystem, the kernel makes it look as if
the part existed somewhere, even though it doesn't. So, even though there is
a multi−megabyte /proc/kcore file, it doesn't take any disk space.

5.9.4. Partitioning a hard disk

There are many programs for creating and removing partitions. Most
operating systems have their own, and it can be a good idea to use each
operating system's own, just in case it does something unusual that the
others can't. Many of the programs are called fdisk, including the Linux one,
or variations thereof. Details on using the Linux fdisk given on its man page.
The cfdisk command is similar to fdisk, but has a nicer (full screen) user
interface.

When using IDE disks, the boot partition (the partition with the bootable
kernel image files) must be completely within the first 1024 cylinders. This is
because the disk is used via the BIOS during boot (before the system goes
into protected mode), and BIOS can't handle more than 1024 cylinders. It is
sometimes possible to use a boot partition that is only partly within the first
1024 cylinders. This works as long as all the files that are read with the BIOS
are within the first 1024 cylinders. Since this is difficult to arrange, it is a very
bad idea to do it; you never know when a kernel update or disk
defragmentation will result in an unbootable system. Therefore, make sure
your boot partition is completely within the first 1024 cylinders.

However, this may no longer be true with newer versions of LILO that support
LBA (Logical Block Addressing). Consult the documentation for your
distribution to see if it has a version of LILO where LBA is supported.

Some newer versions of the BIOS and IDE disks can, in fact, handle disks
with more than 1024 cylinders. If you have such a system, you can forget
about the problem; if you aren't quite sure of it, put it within the first 1024
cylinders.

Each partition should have an even number of sectors, since the Linux
filesystems use a 1 kilobyte block size, i.e., two sectors. An odd number of
sectors will result in the last sector being unused. This won't result in any
problems, but it is ugly, and some versions of fdisk will warn about it.

Changing a partition's size usually requires first backing up everything you

want to save from that partition (preferably the whole disk, just in case),
deleting the partition, creating new partition, then restoring everything to
the new partition. If the partition is growing, you may need to adjust the
sizes (and backup and restore) of the adjoining partitions as well.

Since changing partition sizes is painful, it is preferable to get the partitions

right the first time, or have an effective and easy to use backup system. If
you're installing from a media that does not require much human
intervention (say, from CD−ROM, as opposed to floppies), it is often easy to
play with different configuration at first. Since you don't already have data to
back up, it is not so painful to modify partition sizes several times.

There is a program for MS−DOS, called fips , which resizes an MS−DOS

partition without requiring the backup and restore, but for other filesystems
it is still necessary.

The fips program is included in most Linux distributions. The commercial

partition manager ``Partition

Magic'' also has a similar facility but with a nicer interface. Please do
remember that partitioning is dangerous. Make sure you have a recent
backup of any important data before you try changing partition sizes ``on
the fly''. The program parted can resize other types of partitions as well as
MS−DOS, but sometimes in a limited manner. Consult the parted
documentation before using it, better safe than sorry.

5.9.5. Device files and partitions

Each partition and extended partition has its own device file. The naming
convention for these files is that a partition's number is appended after the
name of the whole disk, with the convention that 1−4 are primary partitions
(regardless of how many primary partitions there are) and number greater
than 5 are logical partitions (regardless of within which primary partition they
reside). For example, /dev/hda1 is the first primary partition on the first IDE
hard disk, and /dev/sdb7 is the third extended partition on the second SCSI
hard disk.

5.10. Filesystems

5.10.1. What are filesystems?

A filesystem is the methods and data structures that an operating system

uses to keep track of files on a disk or partition; that is, the way the files are
organized on the disk. The word is also used to refer to a partition or disk
that is used to store the files or the type of the filesystem. Thus, one might
say ``I have two filesystems'' meaning one has two partitions on which one
stores files, or that one is using the ``extended filesystem'', meaning the
type of the filesystem.

The difference between a disk or partition and the filesystem it contains is

important. A few programs (including, reasonably enough, programs that
create filesystems) operate directly on the raw sectors of a disk or partition;
if there is an existing file system there it will be destroyed or seriously
corrupted. Most programs operate on a filesystem, and therefore won't work
on a partition that doesn't contain one (or that contains one of the wrong
type).

Before a partition or disk can be used as a filesystem, it needs to be

initialized, and the bookkeeping data structures need to be written to the
disk. This process is called making a filesystem.

Most UNIX filesystem types have a similar general structure, although the
exact details vary quite a bit. The central concepts are superblock, inode ,
data block, directory block , and indirection block. The superblock contains
information about the filesystem as a whole, such as its size (the exact
information here depends on the filesystem). An inode contains all
information about a file, except its name. The name is stored in the directory,
together with the number of the inode. A directory entry consists of a
filename and the number of the inode which represents the file. The inode
contains the numbers of several data blocks, which are used to store the
data in the file. There is space only for a few data block numbers in the
inode, however, and if more are needed, more space for pointers to the data
blocks is allocated dynamically. These dynamically allocated blocks are
indirect blocks; the name indicates that in order to find the data block, one
has to find its number in the indirect block first.

UNIX filesystems usually allow one to create a hole in a file (this is done with
the lseek() system call; check the manual page), which means that the
filesystem just pretends that at a particular place in the file there is just zero
bytes, but no actual disk sectors are reserved for that place in the file (this
means that the file will use a bit less disk space). This happens especially
often for small binaries, Linux shared libraries, some databases, and a few
other special cases. (Holes are implemented by storing a special value as the
address of the data block in the indirect block or inode. This special address
means that no data block is allocated for that part of the file, ergo, there is a
hole in the file.)

5.10.2. Filesystems galore

Linux supports several types of filesystems. As of this writing the most

important ones are:

minix

The oldest, presumed to be the most reliable, but quite limited in features
(some time stamps are missing, at most 30 character filenames) and
restricted in capabilities (at most 64 MB per filesystem).

xia

A modified version of the minix filesystem that lifts the limits on the
filenames and filesystem sizes, but does not otherwise introduce new
features. It is not very popular, but is reported to work very well.

ext3

The ext3 filesystem has all the features of the ext2 filesystem. The difference
is, journaling has been added. This improves performance and recovery time
in case of a system crash. This has become more popular than ext2.

ext2
The most featureful of the native Linux filesystems. It is designed to be easily
upwards compatible, so that new versions of the filesystem code do not
require re−making the existing filesystems.

ext

An older version of ext2 that wasn't upwards compatible. It is hardly ever

used in new installations any more, and most people have converted to ext2.
reiserfs

A more robust filesystem. Journaling is used which makes data loss less likely.
Journaling is a mechanism whereby a record is kept of transaction which are
to be performed, or which have been performed. This allows the filesystem
to reconstruct itself fairly easily after damage caused by, for example,
improper shutdowns.

jfs

JFS is a journaled filesystem designed by IBM to to work in high performance

environments> xfs

XFS was originally designed by Silicon Graphics to work as a 64−bit

journaled filesystem. XFS was also designed to maintain high performance
with large files and filesystems.

In addition, support for several foreign filesystems exists, to make it easier to

exchange files with other operating systems. These foreign filesystems work
just like native ones, except that they may be lacking in some usual UNIX
features, or have curious limitations, or other oddities.

msdos

Compatibility with MS−DOS (and OS/2 and Windows NT) FAT filesystems.

umsdos

Extends the msdos filesystem driver under Linux to get long filenames,
owners, permissions, links, and device files. This allows a normal msdos
filesystem to be used as if it were a Linux one, thus removing the need for a
separate partition for Linux. vfat

This is an extension of the FAT filesystem known as FAT32. It supports larger

disk sizes than FAT. Most MS Windows disks are vfat.

iso9660
The standard CD−ROM filesystem; the popular Rock Ridge extension to the
CD−ROM standard that allows longer file names is supported automatically.

nfs

A networked filesystem that allows sharing a filesystem between many

computers to allow easy access to the files from all of them.

smbfs

A networks filesystem which allows sharing of a filesystem with an MS

Windows computer. It is compatible with the Windows file sharing protocols.

hpfs

The OS/2 filesystem.

sysv

SystemV/386, Coherent, and Xenix filesystems.

NTFS

The most advanced Microsoft journaled filesystem providing faster file access
and stability over previous Microsoft filesystems.

The choice of filesystem to use depends on the situation. If compatibility or

other reasons make one of the non−native filesystems necessary, then that
one must be used. If one can choose freely, then it is probably wisest to use
ext3, since it has all the features of ext2, and is a journaled filesystem. For
more information on

filesystems, see Section 5.10.6. You can also read the Filesystems HOWTO
located at http://www.tldp.org/HOWTO/Filesystems−HOWTO.html

There is also the proc filesystem, usually accessible as the /proc directory,
which is not really a filesystem at all, even though it looks like one. The proc
filesystem makes it easy to access certain kernel data structures, such as the
process list (hence the name). It makes these data structures look like a
filesystem, and that filesystem can be manipulated with all the usual file
tools.

5.10.3. Which filesystem should be used?

There is usually little point in using many different filesystems. Currently,

ext3 is the most popular filesystem, because it is a journaled filesystem.
Currently it is probably the wisest choice. Reiserfs is another popular choice
because it to is journaled. Depending on the overhead for bookkeeping
structures, speed, (perceived) reliability, compatibility, and various other
reasons, it may be advisable to use another file system. This needs to be
decided on a case−by−case basis.

A filesystem that uses journaling is also called a journaled filesystem. A

journaled filesystem maintains a log, or journal, of what has happened on a
filesystem. In the event of a system crash, or if your 2 year old son hits the
power button like mine loves to do, a journaled filesystem is designed to use
the filesystem's logs to recreate unsaved and lost data. This makes data loss
much less likely and will likely become a standard feature in Linux
filesystems. However, do not get a false sense of security from this. Like
everything else, errors can arise. Always make sure to back up your data in
the event of an emergency.

See Section 5.10.6 for more details about the features of the different
filesystem types.

5.10.4. Creating a filesystem

Filesystems are created, i.e., initialized, with the mkfs command. There is
actually a separate program for each filesystem type. mkfs is just a front end
that runs the appropriate program depending on the desired filesystem type.
The type is selected with the −t fstype option.

The programs called by mkfs have slightly different command line interfaces.
The common and most important options are summarized below; see the
manual pages for more.−t fstype Select the type of the filesystem−c Search
for bad blocks and initialize the bad block list accordingly.

−l filename Read the initial bad block list from the name file.

There are also many programs written to add specific options when creating
a specific filesystem. For example mkfs.ext3 adds a −b option to allow the
administrator to specify what block size should be used. Be sure to find out if
there is a specific program available for the filesystem type you want to use.
For more information on determining what block size to use please see
Section 5.10.5.

First, the floppy was formatted (the −n option prevents validation, i.e., bad
block checking). Then bad blocks were searched with badblocks, with the
output redirected to a file, bad−blocks. Finally, the filesystem was created,
with the bad block list initialized by whatever badblocks found.
The −c option could have been used with mkfs instead of badblocks and a
separate file. The −c option is more convenient than a separate use of
badblocks, but badblocks is necessary for checking after the filesystem has
been created.

The process to prepare filesystems on hard disks or partitions is the same as

for floppies, except that the formatting isn't needed.

5.10.5. Filesystem block size

The block size specifies size that the filesystem will use to read and write
data. Larger block sizes will help improve disk I/O performance when using
large files, such as databases. This happens because the disk can read or
write data for a longer period of time before having to search for the next
block.

On the downside, if you are going to have a lot of smaller files on that
filesystem, like the /etc, there the potential for a lot of wasted disk space.

For example, if you set your block size to 4096, or 4K, and you create a file
that is 256 bytes in size, it will still consume 4K of space on your harddrive.
For one file that may seem trivial, but when your filesystem contains
hundreds or thousands of files, this can add up.

Block size can also effect the maximum supported file size on some
filesystems. This is because many modern filesystem are limited not by block
size or file size, but by the number of blocks. Therefore you would be using a
"block size * max # of blocks = max block size" formula.

5.10.6. Filesystem comparison

It should be noted that Exabytes, Zettabytes, and Yottabytes are rarely

encountered, if ever. There is a current estimate that the worlds printed
material is equal to 5 Exabytes. Therefore, some of these filesystem
limitations are considered by many as theoretical. However, the filesystem
software has been written with these capabilities.

For more detailed information you can visit

http://en.wikipedia.org/wiki/Comparison_of_file_systems.

5.10.7. Mounting and unmounting

Before one can use a filesystem, it has to be mounted. The operating system
then does various bookkeeping things to make sure that everything works.
Since all files in UNIX are in a single directory tree, the mount operation will
make it look like the contents of the new filesystem are the contents of an
existing subdirectory in some already mounted.

The mount command takes two arguments. The first one is the device file
corresponding to the disk or partition containing the filesystem. The second
one is the directory below which it will be mounted. After these commands
the contents of the two filesystems look just like the contents of the /home
and /usr directories, respectively. One would then say that /dev/hda2 is
mounted on /home'', and similarly for /usr. To look at either filesystem, one
would look at the contents of the directory on which it has been mounted,
just as if it were any other directory. Note the difference between the device
file, /dev/hda2, and the mounted−on directory, /home. The device file gives
access to the raw contents of the disk, the mounted−on directory gives
access to the files on the disk. The mounted−on directory is called the mount
point.

Linux supports many filesystem types. mount tries to guess the type of the
filesystem. You can also use the −t fstype option to specify the type directly;
this is sometimes necessary, since the heuristics mount uses do not always
work.

The mounted−on directory need not be empty, although it must exist. Any
files in it, however, will be inaccessible by name while the filesystem is
mounted. (Any files that have already been opened will still be accessible.
Files that have hard links from other directories can be accessed using those
names.) There is no harm done with this, and it can even be useful. For
instance, some people like to have /tmp and /var/tmp synonymous, and
make /tmp be a symbolic link to /var/tmp. When the system is booted, before
the /var filesystem is mounted, a /var/tmp directory residing on the root
filesystem is used instead. When /var is mounted, it will make the /var/tmp
directory on the root filesystem inaccessible. If /var/tmp didn't exist on the
root filesystem, it would be impossible to use temporary files before
mounting /var.

If you don't intend to write anything to the filesystem, use the −r switch for
mount to do a read−only mount. This will make the kernel stop any attempts
at writing to the filesystem, and will also stop the kernel from updating file
access times in the inodes. Read−only mounts are necessary for unwritable
media, e.g.,CD−ROMs.

The alert reader has already noticed a slight logistical problem. How is the
first filesystem (called the root filesystem, because it contains the root
directory) mounted, since it obviously can't be mounted on another
filesystem? Well, the answer is that it is done by magic. The root filesystem is
magically mounted at boot time, and one can rely on it to always be
mounted. If the root filesystem can't be mounted, the system does not boot.
The name of the filesystem that is magically mounted as root is either
compiled into the kernel, or set using LILO or rdev. For more information, see
the kernel source or the Kernel Hackers' Guide.

The root filesystem is usually first mounted read−only. The startup scripts
will then run fsck to verify its validity, and if there are no problems, they will
re−mount it so that writes will also be allowed. fsck must not be run on a
mounted filesystem, since any changes to the filesystem while fsck is
running will cause trouble. Since the root filesystem is mounted read−only
while it is being checked, fsck can fix any problems without worry, since the
remount operation will flush any metadata that the filesystem keeps in
memory.

On many systems there are other filesystems that should also be mounted
automatically at boot time. These are specified in the /etc/fstab file; see the
fstab man page for details on the format. The details of exactly when the
extra filesystems are mounted depend on many factors, and can be
configured by each administrator if need be; see Chapter 8.

When a filesystem no longer needs to be mounted, it can be unmounted with

umount. umount takes one argument: either the device file or the mount
point. See the man page for further instructions on how to use the command.
It is imperative that you always unmount a mounted floppy. Don't just pop
the floppy out of the drive! Because of disk caching, the data is not
necessarily written to the floppy until you unmount it, so removing the floppy
from the drive too early might cause the contents to become garbled. If you
only read from the floppy, this is not very likely, but if you write, even
accidentally, the result may be catastrophic.

Mounting and unmounting requires super user privileges, i.e., only root can
do it. The reason for this is that if any user can mount a floppy on any
directory, then it is rather easy to create a floppy with, say, a Trojan horse
disguised as /bin/sh, or any other often used program. However, it is often
necessary to allow users to use floppies, and there are several ways to do
this:
Give the users the root password. This is obviously bad security, but is the
easiest solution. It works well if there is no need for security anyway, which is
the case on many non−networked, personal systems.

Use a program such as sudo to allow users to use mount. This is still bad
security, but doesn't directly give super user privileges to everyone. It
requires several seconds of hard thinking on the users' behalf. Furthermore
sudo can be configured to only allow users to execute certain commands.
See the sudo(8), sudoers(5), and visudo(8) manual pages.

Make the users use mtools, a package for manipulating MS−DOS filesystems,
without mounting them. This works well if MS−DOS floppies are all that is
needed, but is rather awkward otherwise. • List the floppy devices and their
allowable mount points together with the suitable options in /etc/fstab. The
columns are: device file to mount, directory to mount on, filesystem type,
options, backup frequency (used by dump), and fsck pass number (to specify
the order in which filesystems should be checked upon boot; 0 means no
check).

The noauto option stops this mount to be done automatically when the
system is started (i.e., it stops mount −a from mounting it). The user option
allows any user to mount the filesystem, and, because of security reasons,
disallows execution of programs (normal or setuid) and interpretation of
device files from the mounted filesystem. After this, any user can mount a
floppy with an msdos filesystem. The floppy can (and needs to, of course) be
unmounted with the corresponding umount command.

If you want to provide access to several types of floppies, you need to give
several mount points. The settings can be different for each mount point.
The "auto" option in the filesystem type column allows the mount command
to query the filesystem and try to determine what type it is itself. This option
won't work on all filesystem types, but works fine on the more common ones.

For MS−DOS filesystems (not just floppies), you probably want to restrict
access to it by using the uid, gid, and umask filesystem options, described in
detail on the mount manual page. If you aren't careful, mounting an
MS−DOS filesystem gives everyone at least read access to the files in it,
which is not a good idea.

5.10.8. Filesystem Security

To be added
This section will describe mount options and how to use them in /etc/fstab to
provide additional system security.

5.10.9. Checking filesystem integrity with fsck

Filesystems are complex creatures, and as such, they tend to be somewhat

error−prone. A filesystem's correctness and validity can be checked using
the fsck command. It can be instructed to repair any minor problems it finds,
and to alert the user if there any unrepairable problems. Fortunately, the
code to implement filesystems is debugged quite effectively, so there are
seldom any problems at all, and they are usually caused by power failures,
failing hardware, or operator errors; for example, by not shutting down the
system properly.

Most systems are setup to run fsck automatically at boot time, so that any
errors are detected (and hopefully corrected) before the system is used. Use
of a corrupted filesystem tends to make things worse: if the data structures
are messed up, using the filesystem will probably mess them up even more,
resulting in more data loss. However, fsck can take a while to run on big
filesystems, and since errors almost never occur if the system has been shut
down properly, a couple of tricks are used to avoid doing the checks in such
cases. The first is that if the file /etc/fastboot exists, no checks are made. The
second is that the ext2 filesystem has a special marker in its superblock that
tells whether the filesystem was unmounted properly after the previous
mount. This allows e2fsck (the version of fsck for the ext2 filesystem) to
avoid checking the filesystem if the flag indicates that the unmount was
done (the assumption being that a proper unmount indicates no problems).
Whether the /etc/fastboot trick works on your system depends on your
startup scripts, but the ext2 trick works every time you use e2fsck. It has to
be explicitly bypassed with an option to e2fsck to be avoided. (See the
e2fsck man page for details on how.)

The automatic checking only works for the filesystems that are mounted
automatically at boot time. Use fsck manually to check other filesystems,
e.g., floppies.

If fsck finds unrepairable problems, you need either in−depth knowledge of

how filesystems work in general, and the type of the corrupt filesystem in
particular, or good backups. The latter is easy (although sometimes tedious)
to arrange, the former can sometimes be arranged via a friend, the Linux
newsgroups and mailing lists, or some other source of support, if you don't
have the know−how yourself. I'd like to tell you more about it, but my lack of
education and experience in this regard hinders me. The debugfs program by
Theodore Ts'o should be useful.

fsck must only be run on unmounted filesystems, never on mounted

filesystems (with the exception of the read−only root during startup). This is
because it accesses the raw disk, and can therefore modify the filesystem
without the operating system realizing it. There will be trouble, if the
operating system is confused.

5.10.10. Checking for disk errors with badblocks

It can be a good idea to periodically check for bad blocks. This is done with
the badblocks command. It outputs a list of the numbers of all bad blocks it
can find. This list can be fed to fsck to be recorded in the filesystem data
structures so that the operating system won't try to use the bad blocks for
storing data.

If badblocks reports a block that was already used, e2fsck will try to move
the block to another place. If the block was really bad, not just marginal, the
contents of the file may be corrupted.

5.10.11. Fighting fragmentation?

When a file is written to disk, it can't always be written in consecutive blocks.

A file that is not stored in consecutive blocks is fragmented. It takes longer to
read a fragmented file, since the disk's read−write head will have to move
more. It is desirable to avoid fragmentation, although it is less of a problem
in a system with a good buffer cache with read−ahead.

Modern Linux filesystem keep fragmentation at a minimum by keeping all

blocks in a file close together, even if they can't be stored in consecutive
sectors. Some filesystems, like ext3, effectively allocate the free block that is
nearest to other blocks in a file. Therefore it is not necessary to worry about
fragmentation in a Linux system.

In the earlier days of the ext2 filesystem, there was a concern over file
fragmentation that lead to the development of a defragmentation program
called, defrag. A copy of it can still be downloaded at
http://www.go.dlr.de/linux/src/defrag−0.73.tar.gz. However, it is HIGHLY
recommended that you NOT use it. It was designed for and older version of
ext2, and has not bee updated since 1998! I only mention it here for
references purposes.
There are many MS−DOS defragmentation programs that move blocks
around in the filesystem to remove fragmentation. For other filesystems,
defragmentation must be done by backing up the filesystem, re−creating it,
and restoring the files from backups. Backing up a filesystem before
defragmenting is a good idea for all filesystems, since many things can go
wrong during the defragmentation.

5.10.12. Other tools for all filesystems

Some other tools are also useful for managing filesystems. df shows the free
disk space on one or more filesystems; du shows how much disk space a
directory and all its files contain. These can be used to hunt down disk space
wasters. Both have manual pages which detail the (many) options which can
be used.

sync forces all unwritten blocks in the buffer cache (see Section 6.6) to be
written to disk. It is seldom necessary to do this by hand; the daemon
process update does this automatically. It can be useful in catastrophes, for
example if update or its helper process bdflush dies, or if you must turn off
power now and can't wait for update to run. Again, there are manual pages.
The man is your very best friend in Linux. Its cousin apropos is also very
useful when you don't know what the name of the command you want is.

5.10.13. Other tools for the ext2/ext3 filesystem

In addition to the filesystem creator (mke2fs) and checker (e2fsck) accessible

directly or via the filesystem type independent front ends, the ext2
filesystem has some additional tools that can be useful. tune2fs adjusts
filesystem parameters. Some of the more interesting parameters are:

A maximal mount count. e2fsck enforces a check when filesystem has been
mounted too many times, even if the clean flag is set. For a system that is
used for developing or testing the system, it might be a good idea to reduce
this limit.

A maximal time between checks. e2fsck can also enforce a maximal time
between two checks, even if the clean flag is set, and the filesystem hasn't
been mounted very often. This can be disabled, however.

Number of blocks reserved for root. Ext2 reserves some blocks for root so
that if the filesystem fills up, it is still possible to do system administration
without having to delete anything. The reserved amount is by default 5
percent, which on most disks isn't enough to be wasteful. However, for
floppies there is no point in reserving any blocks.See the tune2fs manual
page for more information. dumpe2fs shows information about an ext2 or
ext3 filesystem, mostly from the superblock. Below is a sample output. Some
of the information in the output is technical and requires understanding of
how the filesystem works, but much of it is readily understandable even for
lay-admins.

debugfs is a filesystem debugger. It allows direct access to the filesystem

data structures stored on disk and can thus be used to repair a disk that is so
broken that fsck can't fix it automatically. It has also been known to be used
to recover deleted files. However, debugfs very much requires that you
understand what you're doing; a failure to understand can destroy all your
data.

dump and restore can be used to back up an ext2 filesystem. They are ext2
specific versions of the traditional UNIX backup tools. See Section 12.1 for
more information on backups.

5.11. Disks without filesystems

Not all disks or partitions are used as filesystems. A swap partition, for
example, will not have a filesystem on it. Many floppies are used in a
tape−drive emulating fashion, so that a tar (tape archive) or other file is
written directly on the raw disk, without a filesystem. Linux boot floppies
don't contain a filesystem, only the raw kernel.

Avoiding a filesystem has the advantage of making more of the disk usable,
since a filesystem always has some bookkeeping overhead. It also makes the
disks more easily compatible with other systems: for example, the tar file
format is the same on all systems, while filesystems are different on most
systems. You will quickly get used to disks without filesystems if you need
them. Bootable Linux floppies also do not necessarily have a filesystem,
although they may.

One reason to use raw disks is to make image copies of them. For instance, if
the disk contains a partially damaged filesystem, it is a good idea to make an
exact copy of it before trying to fix it, since then you can start again if your
fixing breaks things even more. The first dd makes an exact image of the
floppy to the file floppy−image, the second one writes the image to the
floppy. (The user has presumably switched the floppy before the second
command. Otherwise the command pair is of doubtful usefulness.)
5.12. Allocating disk space

5.12.1. Partitioning schemes

When it comes to partitioning your machine, there is no universally correct

way to do it. There are many factors that must be taken into account
depending on the purpose of the machine.

For a simple workstation with limited disk space, such as a laptop, you may
have as few a 3 partitions. A partition for /, /boot, and swap. However, for
most users this is not a recommended solution.

The traditional way is to have a (relatively) small root filesystem, and

separate partitions for filesystems such as /usr and /home>. Creating a
separate root filesystem if the root filesystem is small and not heavily used,
it is less likely to become corrupt when the system crashes, and therefore
make it easier to recover a crashed system. The reason is to prevent having
the root filesystem get filled and cause a system crash.

When creating your partitioning scheme, there are some things you need to
remember. You cannot create separate partitions for the following directories:
/bin, /etc, /dev, /initrd, /lib, and /sbin. The contents of these directories are
required at bootup and must always be part of the / partition.

It is also recommended that you create separate partitions for /var and /tmp.
This is because both directories typically have data that is constantly
changing. Not creating separate partitions for these filesystems puts you at
risk of having log file fill up our / partition. The problem with having many
partitions is that it splits the total amount of free disk space into many small
pieces. One way to avoid this problem is to use to create Logical Volumes.

5.12.2. Logical Volume Manager (LVM)

Using LVM allows administrators the flexibility to create logical disks that can
be expanded dynamically as more disk space is required.

This is done first by creating partitions with as an 0x8e Linux LVM partition
type. Then the Physical Partitions are added to a Volume Group and broken
up into chunks, or Physical Extents Volume Group. These extends can then
be grouped into Logical Volumes. These Logical Volumes then can be
formatted just like a physical partition. The big difference is that they can be
expanded by adding more extents to them.
Right now, a full discussion of LVM is beyond the scope of this guide.
However, and excellent resource for learning more about LVM can be found
at http://www.tldp.org/HOWTO/LVM−HOWTO.html.

5.12.3. Space requirements

The Linux distribution you install will give some indication of how much disk
space you need for various configurations. Programs installed separately
may also do the same. This will help you plan your disk space usage, but you
should prepare for the future and reserve some extra space for things you
will notice later that you need.

The amount you need for user files depends on what your users wish to do.
Most people seem to need as much space for their files as possible, but the
amount they will live happily with varies a lot. Some people do only light text
processing and will survive nicely with a few megabytes, others do heavy
image processing and will need gigabytes.

By the way, when comparing file sizes given in kilobytes or megabytes and
disk space given in megabytes, it can be important to know that the two
units can be different. Some disk manufacturers like to pretend that a
kilobyte is 1000 bytes and a megabyte is 1000 kilobytes, while all the rest of
the computing world uses 1024 for both factors. Therefore, a 345 MB hard
disk is really a 330 MB hard disk.

Swap space allocation is discussed in Section 6.5.

5.12.4. Examples of hard disk allocation

I used to have a 10 GB hard disk. Now I am using a 30 GB hard disk. I'll

explain how and why I partitioned those disks.

First, I created a /boot partition at 128 MG. This is larger than I will need, and
big enough to give me space if I need it. I created a separate /boot partition
to ensure that this filesystem will never get filled up, and therefore will be
bootable. Then I created a 5 GB /var partition. Since the /var filesystem is
where log files and email is stored I wanted to isolate it from my root
partition. (I have had log files grow overnight and fill my root filesystem in
the past.) Next, I created a 15 GB /home partition. This is handy in the event
of a system crash. If I ever have to re−install Linux from scratch, I can tell
the installation program to not format this partition, and instead remount it
without the data being lost. Finally since I had 512 MG of RAM I created a
1024 MG (or 1 GB) swap partition. This left me with roughly a 9 GB root
filesystem. I using my old 10 GB hard drive, I created an 8 GB /usr partition
and left 2 GB unused. This is incase I need more space in the future.In the
end, my partition tables looked like this:

5.12.5. Adding more disk space for Linux

Adding more disk space for Linux is easy, at least after the hardware has
been properly installed (the hardware installation is outside the scope of this
book). You format it if necessary, then create the partitions and filesystem as
described above, and add the proper lines to /etc/fstab so that it is mounted
automatically.

5.12.6. Tips for saving disk space

The best tip for saving disk space is to avoid installing unnecessary
programs. Most Linux distributions have an option to install only part of the
packages they contain, and by analyzing your needs you might notice that
you don't need most of them. This will help save a lot of disk space, since
many programs are quite large. Even if you do need a particular package or
program, you might not need all of it. For example, some on−line
documentation might be unnecessary, as might some of the Elisp files for
GNU Emacs, some of the fonts for X11, or some of the libraries for
programming.

If you cannot uninstall packages, you might look into compression.

Compression programs such as gzip or zip will compress (and uncompress)
individual files or groups of files. The gzexe system will compress and
uncompress programs invisibly to the user (unused programs are
compressed, then uncompressed as they are used). The experimental
DouBle system will compress all files in a filesystem, invisibly to the
programs that use them. (If you are familiar with products such as Stacker
for MS−DOS or DriveSpace for Windows, the principle is the same.)

Another way to save space is to take special care when formatting you
partitions. Most modern filesystems will allow you to specify the block size.
The block size is chunk size that the filesystem will use to read and write
data. Larger block sizes will help disk I/O performance when using large files,
such as databases. This happens because the disk can read or write data for
a longer period of time before having to search for the next block.

Chapter 6. Memory Management

"Minnet, jag har tappat mitt minne, är jag svensk eller finne, kommer inte
ihåg..." (Bosse Österberg)
A Swedish drinking song, (rough) translation: ``Memory, I have lost my
memory. Am I Swedish or Finnish? I can't remember''

This section describes the Linux memory management features, i.e., virtual
memory and the disk buffer cache. The purpose and workings and the things
the system administrator needs to take into consideration are described.

6.1. What is virtual memory?

Linux supports virtual memory, that is, using a disk as an extension of RAM
so that the effective size of usable memory grows correspondingly. The
kernel will write the contents of a currently unused block of memory to the
hard disk so that the memory can be used for another purpose. When the
original contents are needed again, they are read back into memory. This is
all made completely transparent to the user; programs running under Linux
only see the larger amount of memory available and don't notice that parts
of them reside on the disk from time to time. Of course, reading and writing
the hard disk is slower (on the order of a thousand times slower) than using
real memory, so the programs don't run as fast. The part of the hard disk
that is used as virtual memory is called the swap space.

Linux can use either a normal file in the filesystem or a separate partition for
swap space. A swap partition is faster, but it is easier to change the size of a
swap file (there's no need to repartition the whole hard disk, and possibly
install everything from scratch). When you know how much swap space you
need, you should go for a swap partition, but if you are uncertain, you can
use a swap file first, use the system for a while so that you can get a feel for
how much swap you need, and then make a swap partition when you're
confident about its size.

You should also know that Linux allows one to use several swap partitions
and/or swap files at the same time. This means that if you only occasionally
need an unusual amount of swap space, you can set up an extra swap file at
such times, instead of keeping the whole amount allocated all the time.

A note on operating system terminology: computer science usually

distinguishes between swapping (writing the whole process out to swap
space) and paging (writing only fixed size parts, usually a few kilobytes, at a
time). Paging is usually more efficient, and that's what Linux does, but
traditional Linux terminology talks about swapping anyway.

6.2. Creating a swap space

A swap file is an ordinary file; it is in no way special to the kernel. The only
thing that matters to the kernel is that it has no holes, and that it is prepared
for use with mkswap. It must reside on a local disk, however; it can't reside
in a filesystem that has been mounted over NFS due to implementation
reasons.

The bit about holes is important. The swap file reserves the disk space so
that the kernel can quickly swap out a page without having to go through all
the things that are necessary when allocating a disk sector to a file.

The kernel merely uses any sectors that have already been allocated to the
file. Because a hole in a file means that there are no disk sectors allocated
(for that place in the file), it is not good for the kernel to try to use them.

One good way to create the swap file without holes is through the following
command.

where /extra−swap is the name of the swap file and the size of is given after
the count=. It is best for the size to be a multiple of 4, because the kernel
writes out memory pages, which are 4 kilobytes in size. If the size is not a
multiple of 4, the last couple of kilobytes may be unused.

A swap partition is also not special in any way. You create it just like any
other partition; the only difference is that it is used as a raw partition, that is,
it will not contain any filesystem at all. It is a good idea to mark swap
partitions as type 82 (Linux swap); this will the make partition listings
clearer, even though it is not strictly necessary to the kernel.

After you have created a swap file or a swap partition, you need to write a
signature to its beginning; this contains some administrative information and
is used by the kernel.

Note that the swap space is still not in use yet: it exists, but the kernel does
not use it to provide virtual memory.

You should be very careful when using mkswap, since it does not check that
the file or partition isn't used for anything else. You can easily overwrite
important files and partitions with mkswap! Fortunately, you should only
need to use mkswap when you install your system.

The Linux memory manager limits the size of each swap space to 2 GB. You
can, however, use up to 8 swap spaces simultaneously, for a total of 16GB.

6.3. Using a swap space

An initialized swap space is taken into use with swapon. This command tells
the kernel that the swap space can be used. The path to the swap space is
given as the argument, so to start swapping on a temporary swap file one
might use the following command. Swap spaces can be used automatically
The startup scripts will run the command swapon −a, which will start
swapping on all the swap spaces listed in /etc/fstab. Therefore, the swapon
command is usually used only when extra swap is needed.You can monitor
the use of swap spaces with free.

The first line of output (Mem:) shows the physical memory. The total column
does not show the physical memory used by the kernel, which is usually
about a megabyte. The used column shows the amount of memory used (the
second line does not count buffers). The free column shows completely
unused memory. The shared column shows the amount of memory shared by
several processes; the more, the merrier. The buffers column shows the
current size of the disk buffer cache.

That last line (Swap:) shows similar information for the swap spaces. If this
line is all zeroes, your swap space is not activated.

The same information is available via top, or using the proc filesystem in
file /proc/meminfo. It is currently difficult to get information on the use of a
specific swap space.

A swap space can be removed from use with swapoff. It is usually not
necessary to do it, except for temporary swap spaces. Any pages in use in
the swap space are swapped in first; if there is not sufficient physical
memory to hold them, they will then be swapped out (to some other swap
space). If there is not enough virtual memory to hold all of the pages Linux
will start to thrash; after a long while it should recover, but meanwhile the
system is unusable. You should check (e.g., with free) that there is enough
free memory before removing a swap space from use.

All the swap spaces that are used automatically with swapon −a can be
removed from use with swapoff −a; it looks at the file /etc/fstab to find what
to remove. Any manually used swap spaces will remain in use.

Sometimes a lot of swap space can be in use even though there is a lot of
free physical memory. This can happen for instance if at one point there is
need to swap, but later a big process that occupied much of the physical
memory terminates and frees the memory. The swapped−out data is not
automatically swapped in until it is needed, so the physical memory may
remain free for a long time. There is no need to worry about this, but it can
be comforting to know what is happening.

6.4. Sharing swap spaces with other operating systems

Virtual memory is built into many operating systems. Since they each need it
only when they are running, i.e., never at the same time, the swap spaces of
all but the currently running one are being wasted. It would be more efficient
for them to share a single swap space. This is possible, but can require a bit
of hacking. The Tips−HOWTO at
http://www.tldp.org/HOWTO/Tips−HOWTO.html, which contains some advice
on how to implement this.

6.5. Allocating swap space

Some people will tell you that you should allocate twice as much swap space
as you have physical memory, but this is a bogus rule. Here's how to do it
properly:

Estimate your total memory needs. This is the largest amount of memory
you'll probably need at a time, that is the sum of the memory requirements
of all the programs you want to run at the same time. This can be done by
running at the same time all the programs you are likely to ever be running
at the same time.

For instance, if you want to run X, you should allocate about 8 MB for it, gcc
wants several megabytes (some files need an unusually large amount, up to
tens of megabytes, but usually about four should do), and so on. The kernel
will use about a megabyte by itself, and the usual shells and other small
utilities perhaps a few hundred kilobytes (say a megabyte together). There is
no need to try to be exact, rough estimates are fine, but you might want to
be on the pessimistic side.

Remember that if there are going to be several people using the system at
the same time, they are all going to consume memory. However, if two
people run the same program at the same time, the total memory
consumption is usually not double, since code pages and shared libraries
exist only once.

The free and ps commands are useful for estimating the memory needs.

Add some security to the estimate in step 1. This is because estimates of

program sizes will probably be wrong, because you'll probably forget some
programs you want to run, and to make certain that you have some extra
space just in case. A couple of megabytes should be fine. (It is better to
allocate too much than too little swap space, but there's no need to over−do
it and allocate the whole disk, since unused swap space is wasted space; see
later about adding more swap.) Also, since it is nicer to deal with even
numbers, you can round the value up to the next full megabyte.

Based on the computations above, you know how much memory you'll be
needing in total. So, in order to allocate swap space, you just need to
subtract the size of your physical memory from the total memory needed,
and you know how much swap space you need. (On some versions of UNIX,
you need to allocate space for an image of the physical memory as well, so
the amount computed in step 2 is what you need and you shouldn't do the
subtraction.)

If your calculated swap space is very much larger than your physical memory
(more than a couple times larger), you should probably invest in more
physical memory, otherwise performance will be too low.

It's a good idea to have at least some swap space, even if your calculations
indicate that you need none. Linux uses swap space somewhat aggressively,
so that as much physical memory as possible can be kept free. Linux will
swap out memory pages that have not been used, even if the memory is not
yet needed for anything. This avoids waiting for swapping when it is needed:
the swapping can be done earlier, when the disk is otherwise idle.

Swap space can be divided among several disks. This can sometimes
improve performance, depending on the relative speeds of the disks and the
access patterns of the disks. You might want to experiment with a few
schemes, but be aware that doing the experiments properly is quite difficult.
You should not believe claims that any one scheme is superior to any other,
since it won't always be true.

6.6. The buffer cache

Reading from a disk is very slow compared to accessing (real) memory. In

addition, it is common to read the same part of a disk several times during
relatively short periods of time. For example, one might first read an e−mail
message, then read the letter into an editor when replying to it, then make
the mail program read it again when copying it to a folder. Or, consider how
often the command ls might be run on a system with many users. By reading
the information from disk only once and then keeping it in memory until no
longer needed, one can speed up all but the first read. This is called disk
buffering, and the memory used for the purpose is called the buffer cache.
Since memory is, unfortunately, a finite, nay, scarce resource, the buffer
cache usually cannot be big enough (it can't hold all the data one ever wants
to use). When the cache fills up, the data that has been unused for the
longest time is discarded and the memory thus freed is used for the new
data.

Disk buffering works for writes as well. On the one hand, data that is written
is often soon read again (e.g., a source code file is saved to a file, then read
by the compiler), so putting data that is written in the cache is a good idea.
On the other hand, by only putting the data into the cache, not writing it to
disk at once, the program that writes runs quicker. The writes can then be
done in the background, without slowing down the other programs.

Most operating systems have buffer caches (although they might be called
something else), but not all of them work according to the above principles.
Some are write−through: the data is written to disk at once (it is kept in the
cache as well, of course). The cache is called write−back if the writes are
done at a later time. Write−back is more efficient than write−through, but
also a bit more prone to errors: if the machine crashes, or the power is cut at
a bad moment, or the floppy is removed from the disk drive before the data
in the cache waiting to be written gets written, the changes in the cache are
usually lost. This might even mean that the filesystem (if there is one) is not
in full working order, perhaps because the unwritten data held important
changes to the bookkeeping information.

Because of this, you should never turn off the power without using a proper
shutdown procedure or remove a floppy from the disk drive until it has been
unmounted (if it was mounted) or after whatever program is using it has
signaled that it is finished and the floppy drive light doesn't shine anymore.
The sync command flushes the buffer, i.e., forces all unwritten data to be
written to disk, and can be used when one wants to be sure that everything
is safely written. In traditional UNIX systems, there is a program called
update running in the background which does a sync every 30 seconds, so it
is usually not necessary to use sync. Linux has an additional daemon,
bdflush, which does a more imperfect sync more frequently to avoid the
sudden freeze due to heavy disk I/O that sync sometimes causes.

Under Linux, bdflush is started by update. There is usually no reason to worry

about it, but if bdflush happens to die for some reason, the kernel will warn
about this, and you should start it by hand (/sbin/update).
The cache does not actually buffer files, but blocks, which are the smallest
units of disk I/O (under Linux, they are usually 1 KB). This way, also
directories, super blocks, other filesystem bookkeeping data, and
non−filesystem disks are cached.

The effectiveness of a cache is primarily decided by its size. A small cache is

next to useless: it will hold so little data that all cached data is flushed from
the cache before it is reused. The critical size depends on how much data is
read and written, and how often the same data is accessed. The only way to
know is to experiment.

If the cache is of a fixed size, it is not very good to have it too big, either,
because that might make the free memory too small and cause swapping
(which is also slow). To make the most efficient use of real memory, Linux
automatically uses all free RAM for buffer cache, but also automatically
makes the cache smaller when programs need more memory.

Under Linux, you do not need to do anything to make use of the cache, it
happens completely automatically. Except for following the proper
procedures for shutdown and removing floppies, you do not need to worry
about it.

Chapter 7. System Monitoring

"That's Hall Monitor to you!"Spongebob Squarepants One of the most

important responsibilities a system administrator has, is monitoring their
systems. As a system administrator you'll need the ability to find out what is
happening on your system at any given time. Whether it's the percentage of
system's resources currently used, what commands are being run, or who is
logged on. This chapter will cover how to monitor your system, and in some
cases, how to resolve problems that may arise.

When a performance issue arises, there are 4 main areas to consider: CPU,
Memory, Disk I/O, and Network. The ability to determine where the
bottleneck is can save you a lot of time.

7.1. System Resources

Being able to monitor the performance of your system is essential. If system

resources become to low it can cause a lot of problems. System resources
can be taken up by individual users, or by services your system may host
such as email or web pages. The ability to know what is happening can help
determine whether system upgrades are needed, or if some services need to
be moved to another machine.
7.1.1. The top command.

The most common of these commands is top. The top will display a
continually updating report of system resource usage. The top portion of the
report lists information such as the system time, uptime, CPU usage, physical
ans swap memory usage, and number of processes. Below that is a list of the
processes sorted by CPU utilization.

You can modify the output of top while is is running. If you hit an i, top will no
longer display idle processes. Hit i again to see them again. Hitting M will
sort by memory usage, S will sort by how long they processes have been
running, and P will sort by CPU usage again.

In addition to viewing options, you can also modify processes from within the
top command. You can use u to view processes owned by a specific user, k to
kill processes, and r to renice them.

For more in−depth information about processes you can look in the /proc
filesystem. In the /proc filesystem you will find a series of sub−directories
with numeric names. These directories are associated with the processes ids
of currently running processes. In each directory you will find a series of files
containing information about the process.

7.1.2. The iostat command.

The iostat will display the current CPU load average and disk I/O information.
This is a great command to monitor . For 2.4 kernels the devices is names
using the device's major and minor number. In this case the device listed
is /dev/hda. To have iostat print this out for you, use the −x. The iostat man
page contains a detailed explanation of what each of these columns mean.

7.1.3. The ps command

The ps will provide you a list of processes currently running. There is a wide
variety of options that this command gives you.

A common use would be to list all processes currently running. To do this you
would use the ps −ef command. (Screen output from this command is too
large to include, the following is only a partial output.)

The first column shows who owns the process. The second column is the
process ID. The Third column is the parent process ID. This is the process
that generated, or started, the process. The forth column is the CPU usage
(in percent). The fifth column is the start time, of date if the process has
been running long enough. The sixth column is the tty associated with the
process, if applicable. The seventh column is the cumulitive CPU usage (total
amount of CPU time is has used while running). The eighth column is the
command itself.

With this information you can see exacly what is running on your system and
kill run−away processes, or those that are causing problems.

7.1.4. The vmstat command

The vmstat command will provide a report showing statistics for system
processes, memory, swap, I/O, and the CPU. These statistics are generated
using data from the last time the command was run to the present. In the
case of the command never being run, the data will be from the last reboot
until the present

7.1.5. The lsof command

The lsof command will print out a list of every file that is in use. Since Linux
considers everythihng a file, this list can be very long. However, this
command can be useful in diagnosing problems. An example of this is if you
wish to unmount a filesystem, but you are being told that it is in use. You
could use this command and grep for the name of the filesystem to see who
is using it. Or suppose you want to see all files in use by a particular process.
To do this you would use lsof −p −processid−.

7.1.6. Finding More Utilities

To learn more about what command line tools are available, Chris Karakas
has wrote a reference guide titled GNU/Linux Command−Line Tools
Summary. It's a good resource for learning what tools are out there and how
to do a number of tasks.

7.2. Filesystem Usage

Many reports are currently talking about how cheap storage has gotten, but
if you're like most of us it isn't cheap enough. Most of us have a limited
amount of space, and need to be able to monitor it and control how it's used.

7.2.1. The df command

The df is the simplest tool available to view disk usage. Simply type in df and
you'll be shown disk usage for all your mounted filesystems in 1K blocks. You
can also use the −h to see the output in "human−readable" format. This will
be in K, Megs, or Gigs depending on the size of the filesystem. Alternately,
you can also use the −B to specify block size.
In addition to space usage, you could use the −i option to view the number
of used and available inodes.

7.2.2. The du command

Now that you know how much space has been used on a filesystem how can
you find out where that data is? To view usage by a directory or file you can
use du. Unless you specify a filename du will act recursively.

7.2.3. Quotas

For more information about quotas you can read The Quota .

7.3. Monitoring Users

Just because you're paranoid doesn't mean they AREN'T out to get you.From
time to time there are going to be occasions where you will want to know
exactly what people are doing on your system. Maybe you notice that a lot of
RAM is being used, or a lot of CPU activity. You are going to want to see who
is on the system, what they are running, and what kind of resources they are
using.

7.3.1. The who command

The easiest way to see who is on the system is to do a who or w. The −−>
who is a simple tool that lists out who is logged −−> on the system and
what port or terminal they are logged on at. 7.3.2. The ps command −again!

In the previous section we can see that user aweeks is logged onto both
pts/1 and pts/2, but what if we want to see what they are doing? We could to
a ps −u aweeks and get the following output. From this we can see that the
user is doing a ps ssh. This is a much more consolidated use of the ps than
discussed previously.

7.3.3. The w command

Even easier than using the who and ps −u commands is to use the w. w will
print out not only who is on the system, but also the comma. From this we
can see that I have a kde session running, I'm working in this document and
have another terminal open sitting idle at a bash prompt.

7.3.4. The skill command

To Be Added

7.3.5. nice and renice

To Be Added

Chapter 8. Boots And Shutdowns

This section explains what goes on when a Linux system is brought up and
taken down, and how it should be done properly. If proper procedures are not
followed, files might be corrupted or lost.

8.1. An overview of boots and shutdowns

The act of turning on a computer system and causing its operating system to
be loaded is called booting. The name comes from an image of the computer
pulling itself up from its bootstraps, but the act itself slightly more realistic.

During bootstrapping, the computer first loads a small piece of code called
the bootstrap loader, which in turn loads and starts the operating system.
The bootstrap loader is usually stored in a fixed location on a hard disk or a
floppy. The reason for this two step process is that the operating system is
big and complicated, but the first piece of code that the computer loads
must be very small (a few hundred bytes), to avoid making the firmware
unnecessarily complicated.

Different computers do the bootstrapping differently. For PCs, the computer

(its BIOS) reads in the first sector (called the boot sector) of a floppy or hard
disk. The bootstrap loader is contained within this sector. It loads the
operating system from elsewhere on the disk (or from some other place).

After Linux has been loaded, it initializes the hardware and device drivers,
and then runs init. init starts other processes to allow users to log in, and do
things. The details of this part will be discussed below.

In order to shut down a Linux system, first all processes are told to terminate
(this makes them close any files and do other necessary things to keep
things tidy), then filesystems and swap areas are unmounted, and finally a
message is printed to the console that the power can be turned off. If the
proper procedure is not followed, terrible things can and will happen; most
importantly, the filesystem buffer cache might not be flushed, which means
that all data in it is lost and the filesystem on disk is inconsistent, and
therefore possibly unusable.

8.2. The boot process in closer look

When a PC is booted, the BIOS will do various tests to check that everything
looks all right, and will then start the actual booting. This process is called
the power on self test , or POST for short. It will choose a disk drive (typically
the first floppy drive, if there is a floppy inserted, otherwise the first hard
disk, if one is installed in the computer; the order might be configurable,
however) and will then read its very first sector. This is called the boot
sector; for a hard disk, it is also called the master boot record, since a hard
disk can contain several partitions, each with their own boot sectors.

The boot sector contains a small program (small enough to fit into one
sector) whose responsibility is to read the actual operating system from the
disk and start it. When booting Linux from a floppy disk, the boot sector
contains code that just reads the first few hundred blocks (depending on the
actual kernel size, of course) to a predetermined place in memory. On a Linux
boot floppy, there is no filesystem, the kernel is just stored in consecutive
sectors, since this simplifies the boot process. It is possible, however, to boot
from a floppy with a filesystem, by using LILO, the LInux LOader, or GRUB,
the GRand Unifying Bootloader.

When booting from the hard disk, the code in the master boot record will
examine the partition table (also in the master boot record), identify the
active partition (the partition that is marked to be bootable), read the boot
sector from that partition, and then start the code in that boot sector. The
code in the partition's boot sector does what a floppy disk's boot sector does:
it will read in the kernel from the partition and start it. The details vary,
however, since it is generally not useful to have a separate partition for just
the kernel image, so the code in the partition's boot sector can't just read the
disk in sequential order, it has to find the sectors wherever the filesystem
has put them. There are several ways around this problem, but the most
common way is to use a boot loader like LILO or GRUB. (The details about
how to do this are irrelevant for this discussion, however; see the LILO or
GRUB documentation for more information; it is most thorough.)

When booting, the bootloader will normally go right ahead and read in and
boot the default kernel. It is also possible to configure the boot loader to be
able to boot one of several kernels, or even other operating systems than
Linux, and it is possible for the user to choose which kernel or operating
system is to be booted at boot time. LILO, for example, can be configured so
that if one holds down the alt, shift, or ctrl key at boot time (when LILO is
loaded), LILO will ask what is to be booted and not boot the default right
away. Alternatively, the bootloader can be configured so that it will always
ask, with an optional timeout that will cause the default kernel to be booted.
It is also possible to give a kernel command line argument, after the name of
the kernel or operating system. For a list of possible options you can read
http://www.tldp.org/HOWTO/BootPrompt−HOWTO.html.

Booting from floppy and from hard disk have both their advantages, but
generally booting from the hard disk is nicer, since it avoids the hassle of
playing around with floppies. It is also faster. Most Linux distributions will
setup the bootloader for you during the install process.

After the Linux kernel has been read into the memory, by whatever means,
and is started for real, roughly the following things happen:

The Linux kernel is installed compressed, so it will first uncompress itself. The
beginning of the kernel image contains a small program that does this.

If you have a super−VGA card that Linux recognizes and that has some
special text modes (such as 100 columns by 40 rows), Linux asks you which
mode you want to use. During the kernel compilation, it is possible to preset
a video mode, so that this is never asked. This can also be done with LILO,
GRUB or rdev.

After this, the kernel checks what other hardware there is (hard disks,
floppies, network adapters, etc), and configures some of its device drivers
appropriately; while it does this, it outputs messages about its findings. For
example, when I boot, I it looks like this The exact texts are different on
different systems, depending on the hardware, the version of Linux being
used, and how it has been configured.

Then the kernel will try to mount the root filesystem. The place is
configurable at compilation time, or any time with rdev or the bootloader.
The filesystem type is detected automatically. If the mounting of the root
filesystem fails, for example because you didn't remember to include the
corresponding filesystem driver in the kernel, the kernel panics and halts the
system (there isn't much it can do, anyway).The root filesystem is usually
mounted read−only (this can be set in the same way as the place). This
makes it possible to check the filesystem while it is mounted; it is not a good
idea to check a filesystem that is mounted read−write.

After this, the kernel starts the program init (located in /sbin/init) in the
background (this will always become process number 1). init does various
startup chores. The exact things it does depends on how it is configured; see
Section 2.3.1 for more information (not yet written). It will at least start some
essential background daemons.
init then switches to multi−user mode, and starts a getty for virtual consoles
and serial lines. getty is the program which lets people log in via virtual
consoles and serial terminals. init may also start some other programs,
depending on how it is configured.

After this, the boot is complete, and the system is up and running normally.

8.2.1. A Word About Bootloaders

To be added

This section will give an overview of the difference between GRUB and
LILO.For more information on LILO, you can read
http://www.tldp.org/HOWTO/LILO.html For more information on GRUB, you
can visit http://www.gnu.org/software/grub/grub.html

8.3. More about shutdowns

It is important to follow the correct procedures when you shut down a Linux
system. If you fail do so, your filesystems probably will become trashed and
the files probably will become scrambled. This is because Linux has a disk
cache that won't write things to disk at once, but only at intervals. This
greatly improves performance but also means that if you just turn off the
power at a whim the cache may hold a lot of data and that what is on the
disk may not be a fully working filesystem (because only some things have
been written to the disk).

Another reason against just flipping the power switch is that in a

multi−tasking system there can be lots of things going on in the background,
and shutting the power can be quite disastrous. By using the proper
shutdown sequence, you ensure that all background processes can save
their data.

The command for properly shutting down a Linux system is shutdown. It is

usually used in one of two ways.

If you are running a system where you are the only user, the usual way of
using shutdown is to quit all running programs, log out on all virtual
consoles, log in as root on one of them (or stay logged in as root if you
already are, but you should change to root's home directory or the root
directory, to avoid problems with unmounting), then give the command
shutdown −h now (substitute now with a plus sign and a number in minutes
if you want a delay, though you usually don't on a single user system).
Alternatively, if your system has many users, use the command shutdown
−h +time message, where time is the time in minutes until the system is
halted, and message is a short explanation of what shuting down.

This will warn everybody that the system will shut down in ten minutes, and
that they'd better get lost or lose data. The warning is printed to every
terminal on which someone is logged in, including all.

We will install a new disk. System should be back on−line in three hours.
The system is going DOWN for system halt in 10 minutes. The warning is
automatically repeated a few times before the boot, with shorter and shorter
intervals as the time runs out.

When the real shutting down starts after any delays, all filesystems (except
the root one) are unmounted, user processes (if anybody is still logged in)
are killed, daemons are shut down, all filesystem are unmounted, and
generally everything settles down. When that is done, init prints out a
message that you can power down the machine. Then, and only then, should
you move your fingers towards the power switch.

Sometimes, although rarely on any good system, it is impossible to shut

down properly. For instance, if the kernel panics and crashes and burns and
generally misbehaves, it might be completely impossible to give any new
commands, hence shutting down properly is somewhat difficult, and just
about everything you can do is hope that nothing has been too severely
damaged and turn off the power. If the troubles are a bit less severe (say,
somebody hit your keyboard with an axe), and the kernel and the update
program still run normally, it is probably a good idea to wait a couple of
minutes to give update a chance to flush the buffer cache, and only cut the
power after that.

In the old days, some people like to shut down using the command sync
three times, waiting for the disk I/O to stop, then turn off the power. If there
are no running programs, this is equivalent to using shutdown. However, it
does not unmount any filesystems and this can lead to problems with the
ext2fs ``clean filesystem'' flag. The triple−sync method is not recommended.

(In case you're wondering: the reason for three syncs is that in the early days
of UNIX, when the commands were typed separately, that usually gave
sufficient time for most disk I/O to be finished.)

8.4. Rebooting
Rebooting means booting the system again. This can be accomplished by
first shutting it down completely, turning power off, and then turning it back
on. A simpler way is to ask shutdown to reboot the system, instead of merely
halting it. This is accomplished by using the −r option to shutdown, for
example, by giving the command shutdown −r now.

Most Linux systems run shutdown −r now when ctrl−alt−del is pressed on

the keyboard. This reboots the system. The action on ctrl−alt−del is
configurable, however, and it might be better to allow for some delay before
the reboot on a multiuser machine. Systems that are physically accessible to
anyone might even be configured to do nothing when ctrl−alt−del is
pressed.

8.5. Single user mode

The shutdown command can also be used to bring the system down to single
user mode, in which no one can log in, but root can use the console. This is
useful for system administration tasks that can't be done while the system is
running normally.

8.6. Emergency boot floppies

It is not always possible to boot a computer from the hard disk. For example,
if you make a mistake in configuring LILO, you might make your system
unbootable. For these situations, you need an alternative way of booting that
will always work (as long as the hardware works). For typical PCs, this means
booting from the floppy drive.

Most Linux distributions allow one to create an emergency boot floppy during
installation. It is a good idea to do this. However, some such boot disks
contain only the kernel, and assume you will be using the programs on the
distribution's installation disks to fix whatever problem you have. Sometimes
those programs aren't enough; for example, you might have to restore some
files from backups made with software not on the installation disks.

Thus, it might be necessary to create a custom root floppy as well. The

Bootdisk HOWTO by Graham Chapman contains instructions for doing this.
You can find this HOWTO at
http://www.tldp.org/HOWTO/Bootdisk−HOWTO/index.html. You must, of
course, remember to keep your emergency boot and root floppies up to date.

You can't use the floppy drive you use to mount the root floppy for anything
else. This can be inconvenient if you only have one floppy drive. However, if
you have enough memory, you can configure your boot floppy to load the
root disk to a ramdisk (the boot floppy's kernel needs to be specially
configured for this). Once the root floppy has been loaded into the ramdisk,
the floppy drive is free to mount other disks.

Chapter 9. init

"Uuno on numero yksi" (Slogan for a series of Finnish movies.)

This chapter describes the init process, which is the first user level process
started by the kernel. init has many important duties, such as starting getty
(so that users can log in), implementing run levels, and taking care of
orphaned processes. This chapter explains how init is configured and how
you can make use of the different run levels.

9.1. init comes first

init is one of those programs that are absolutely essential to the operation of
a Linux system, but that you still can mostly ignore. A good Linux distribution
will come with a configuration for init that will work for most systems, and on
these systems there is nothing you need to do about init. Usually, you only
need to worry about init if you hook up serial terminals, dial−in (not
dial−out) modems, or if you want to change the default run level.

When the kernel has started itself (has been loaded into memory, has started
running, and has initialized all device drivers and data structures and such),
it finishes its own part of the boot process by starting a user level program,
init. Thus, init is always the first process (its process number is always 1).

The kernel looks for init in a few locations that have been historically used for
it, but the proper location for it (on a Linux system) is /sbin/init. If the kernel
can't find init, it tries to run /bin/sh, and if that also fails, the startup of the
system fails.

When init starts, it finishes the boot process by doing a number of

administrative tasks, such as checking filesystems, cleaning up /tmp, starting
various services, and starting a getty for each terminal and virtual console
where users should be able to log in (see Chapter 10).

After the system is properly up, init restarts getty for each terminal after a
user has logged out (so that the next user can log in). init also adopts orphan
processes: when a process starts a child process and dies before its child, the
child immediately becomes a child of init. This is important for various
technical reasons, but it is good to know it, since it makes it easier to
understand process lists and process tree graphs. There are a few variants of
init available. Most Linux distributions use sysvinit (written by Miquel van
Smoorenburg), which is based on the System V init design. The BSD versions
of Unix have a different init. The primary difference is run levels: System V
has them, BSD does not (at least traditionally). This difference is not
essential. We'll look at sysvinit only.

9.2. Configuring init to start getty: the /etc/inittab file

When it starts up, init reads the /etc/inittab configuration file. While the
system is running, it will re−read it, if sent the HUP signal (kill −HUP 1); this
feature makes it unnecessary to boot the system to make changes to the init
configuration take effect. The /etc/inittab file is a bit complicated. We'll start
with the simple case of configuring getty lines. Lines in /etc/inittab consist of
four cross field limits.