File Structures and Manipulation of Files in UNIX: Ics 2305: Systems Programming
File Structures and Manipulation of Files in UNIX: Ics 2305: Systems Programming
Files in UNIX
July 3, 2025 2
UNIX FILES
You interact with a UNIX system through a command interpreter called
the shell.
• The power of UNIX lies in combining these commands in the same
way the English language lets you combine words to generate
meaningful idea.
THE UNIX ARCHITECTURE
• The success of UNIX, according to Thompson and Ritchie, “lies not so
much in new inventions but rather in the full exploitation of a carefully
selected fertile ideas, and especially in showing that they can be keys to
the implementation of a small and yet powerful operating system”.
UNIX architecture has three important agencies
• Division of labor: Kernel and shell
• The file and process
• The system calls
July 3, 2025 3
Division of labor: Kernel and shell
The fertile ideas in the development of UNIX has two agencies – kernel and
shell.
• The kernel interacts with the machine's hardware.
• The shell interacts with the user.
The Kernel
The core of the operating system - a collection of routines mostly written in
C. It is loaded into memory when the system is booted and communicates
directly with the hardware.
• User programs (the applications) that need to access the hardware use
the services of the kernel, which performs the job on the user's behalf.
• These programs access the kernel through a set of functions called
system calls.
• It manages the system's memory, schedules processes, decides their
priorities and so on.
• a programs gateway to the computer's resources.
July 3, 2025 4
The Shell
• Computers don't have any capability of translating commands into
action.
• That requires a command interpreter, also called as the shell.
• Shell is actually interface between the user and the kernel.
• The shell accepts commands from user, if require rebuilds a user
command, and finally communicates with the kernel to see that the
command is executed.
Example:
• echo CTO Geoffrey
#Shell rebuilds echo command by removing multiple spaces.
CTO Geoffrey
The following figure shows the kernel-shell Relationship:
July 3, 2025 5
The file and process
• Two simple entities support the UNIX – the file and the process.
• “Files have places and processes have life”
The File
• A file is just an array of bytes and can contain virtually anything.
• Every file in UNIX is part of the one file structure provided by UNIX.
• UNIX considers directories and devices as members of the file system.
The Process
• The process is the name given to the file when it is executed as a
program (Process is program under execution).
• We can say process is an “time image” of an executable file.
• We also treat process as a living organism which have parents, children
and are born and die.
July 3, 2025 6
The System Calls
• The UNIX system-comprising the kernel, shell and applications-is
written in C.
• Though there are several commands that use functions called system
calls to communicate with the kernel.
• All UNIX flavors have one thing in common – they use the same system
calls.
July 3, 2025 7
FEATURES OF UNIX
• A Multiuser System
• A Multitasking System
• The building-block approach
• The UNIX toolkit
• Pattern Matching
• Programming Facility
• Documentation
July 3, 2025 8
A Multiuser System
UNIX is a multiprogramming system, it permits multiple programs to run and
compete for the attention of the CPU. This can happen in two ways:
• Multiple users can run separate jobs
• A single user can also run multiple jobs
A Multitasking System
• A single user can also run multiple tasks concurrently.
• UNIX is a multitasking system.
• It is usual for a user to edit a file, print another one on the printer, send
email to a friend and browse www - all without leaving any of applications.
• The kernel is designed to handle a user's multiple needs.
• In a multitasking environment, a user sees one job running in the
foreground; the rest run in the background.
• User can switch jobs between background and foreground, suspend, or
even terminate them.
July 3, 2025 9
The Building-block Approach
UNIX offers the | (filters) to combine various simple tools to carry out
complex jobs. Example:
• $ cat note #cat displays the file contents
• WELCOME TO HIT
• $ cat note | wc #wc counts number of lines, words & characters in the file
The UNIX Toolkit
• UNIX offers facility to add and remove many applications as and when
required.
• Tools include general purpose tools, text manipulation tools, compilers,
interpreters, networked applications and system administration tools.
Pattern Matching
• UNIX features very sophisticated pattern matching features.
• Example: The * (zero or more occurrences of characters) is a special
character used by system to indicate that it can match a number of
filenames.
July 3, 2025 10
Programming Facility
• It has all the necessary ingredients, like control structures, loops and
variables, that establish powerful programming language.
• These features are used to design shell scripts – programs that can also
invoke UNIX commands.
• Many of the system's functions can be controlled and automated by
using these shell scripts.
Documentation
• The principal on-line help facility available is the man command, which
remains the most important references for commands and their
configuration files.
• Apart from the man documentation, there's a vast ocean of UNIX
resources available on the Internet.
July 3, 2025 11
LOCATING COMMANDS
• The UNIX is command based system i.e.,- things happens because the
user enters commands in.
• UNIX commands are seldom more than four characters long.
• All UNIX commands are single words like – cat, ls, pwd, date, mkdir,
rmdir, cd, grep etc.
• The command names are all in lowercase.
Example:
$ LS bash: LS: command not found
• All UNIX commands are files containing programs, mainly written in C.
• All UNIX commands(files) are stored in directories(folders).
• If you want to know the location of executable program (or command),
use type command-
July 3, 2025 12
Example:
• $ type date date is /bin/date
• When you execute date command, the shell locates this file in
the /bin directory and makes arrangements to execute it.
• The PATH
• The sequence of directories that the shell searches to look for
a command is specified in its own PATH variable.
• Example:
• $ echo $PATH /bin: /usr/bin: /usr/local/bin: /usr/ccs/bin:
/usr/local/java/bin:
July 3, 2025 13
INTERNAL AND EXTERNAL COMMANDS
• When the shell execute command(file) from its own set of built-in
commands that are not stored as separate files in /bin directory, it is
called internal command.
• If the command (file) has an independence existence in the /bin
directory, it is called external command.
Examples:
• $ type echo # echo is an internal command echo is shell built-in
July 3, 2025 14
command arguments
• Commands and arguments have to be separated by spaces
or tabs to enable the system to interpret them as words.
• UNIX arguments range from the simple to the complex.
• Arguments may consist of options, expressions, instructions
and filenames etc.
Filename Arguments
• Many UNIX commands use a filename as argument so the
command can take input from the file.
• If a command uses a filename as argument, it will generally
be its last argument.
• It's also quite common to see many commands working with
multiple filenames as arguments.
July 3, 2025 15
Filename Arguments
The complete line is then fed to the shell as its input for interpretation
and execution.
Examples:
• $ ls -lat chap01 chap02 chap03 # Multiple filenames as
arguments
• $ rm chap01 chap02
• $ cp chap01 chap01.bak
Exceptions
• There are some commands that don't accept any arguments.
• The ls command can run without arguments (ls), with only options (ls -l)
and also with only filenames like- (ls chap01 chap02), Examples:
• $ pwd # pwd prints the current working directory
• /root
• $ who # who lists currently logged in users
• mahesh tty7 2013-01-30 09:08 mahesh pts/1 2013-01-30 10:20 (:0)
July 3, 2025 16
date: DISPLAYING THE SYSTEM DATE
• One can display the current date with the date command, which
shows the date and time to the nearest second:
• $ date
• Mon Sep 4 16:40:02 IST 2017
July 3, 2025 17
DISPLAYING THE SYSTEM DATE
• Or
• the month name name:
• $ date +%h
• Aug
• Or
• You can combine them in one command:
• $ date + “%h %m”
• Aug 08
• There are many other format specifiers, and the useful ones are listed
below:
d – The day of month (1 - 31)
y – The last two digits of the year.
H, M and S – The hour, minute and second, respectively.
D – The date in the format mm/dd/yy
T – The time in the format hh:mm:ss
July 3, 2025 19
Displaying the Message
• printf also accepts all escape sequences used by echo, but
unlike echo, it doesn‟t automatically insert newline unless the
\n is used explicitly. printf also uses formatted strings in the
same way the C language function of the same name uses
them:
• $ printf "My current shell is %s\n" $SHELL
• My current shell is /bin/bash
• $_
July 3, 2025 20
String~Character Formating
• The %s format string acts as a placeholder for the value of
$SHELL, and printf replaces %s with the value of $SHELL. %s is
the standard format used for printing strings. printf uses many of the
formats used by C‟s printf function. Here ara some of the commonly
used ones:
• %s – String
• %30s – As above but printed in a space 30 characters wide
• %d – Decimal integer
• %6d - As above but printed in a space 30 characters wide
• %o – Octal integer
• %x – Hexadecimal integer
• %f – Floating point number
July 3, 2025 21
who: WHO ARE THE USERS
• UNIX maintains an account of list of all users
logged on to the system. The who command
displays an informative listing of these users:
July 3, 2025 22
File Types in UNIX
• Regular Files: Contain data, text,
or program code.
• Directories: Special files that store
information about other files.
• Symbolic Links: Pointers to other
files or directories.
• Device Files: Represent hardware
devices as files (character and
block devices).
• Named Pipes: Special files for
inter-process communication.
• Sockets: Used for network
communication.
July 3, 2025 23
Types of files in unix system_Short description
• Ordinary Files - contains data, text, or program instructions. Used to store your
information, such as some text you have written or an image you have drawn. In
long-format output of ls -l, this type of file is specified by the “-” symbol.
• Directories - store both special and ordinary files. Contains an entry for every file
and subdirectory that it houses. All files are descendants of the root directory,
( named / ) located at the top of the tree. In long-format output of ls –l , this type
of file is specified by the “d” symbol.
• Special Files - Used to represent a real physical device such as a printer, tape
drive or terminal, used for Input/Output (I/O) operations. In long-format output of
ls -l, character special files are marked by the “c” symbol. In long-format output of
ls -l, block special files are marked by the “b” symbol.
• Pipes – Pipe acts as a temporary file which only exists to hold data from one
command until it is read by another. For example: who | wc -l In long-format
output of ls –l , named pipes are marked by the “p” symbol.
• Sockets - A Unix socket (or Inter-process communication socket) is a special file
which allows for advanced inter-process communication. A Unix Socket is used in
a client-server application framework. In essence, it is a stream of data, very
similar to network stream. output of ls -l, Unix sockets are marked by “s” symbol.
• Symbolic Link - contains a text form of the path to the file it references. output of
ls –l , Symbolic link are marked by the “l” symbol
July 3, 2025 CSE 373 Wi05 - Introduction 24
The Directory Structure
• All information, be it
a program source, lett
ers, programs,
or system
applications, are
all heldin “files”. These
files are grouped
together in
“directories”. The
directories are then
arrangedas a “tree”,
starting from the
system
• root
• directory. This is
shown schematically
is figure
July 3, 2025 Schematic of part-of the UNIX file system. 25
Directory Structure
• For example the file that contained the source code for the
solutionto yourfirst checkpoint maybe located in:
• /Home/s0312345/sci-prog/checkpoints/CheckPoint1.java
• while the JAVA compiler javac is located in: /usr/java/bin/javac
The structure is divided into user and system files, being the files that
belong to the users of thesystem and the files that are part of the
general system. On this system the users file are under /Home, while
the system one are mostly under /usr (the name is historical!
July 3, 2025 28
File Manipulation Commands
UNIX provides several commands for managing files:
Creating and Viewing Files
• touch filename – Create an empty file.
• cat filename – Display file contents.
• less filename – View file content one page at a time.
• more filename – Similar to less, but only scrolls forward.
• head -n filename – View the first n lines.
• tail -n filename – View the last n lines.
July 3, 2025 29
Copying, Moving and Deleting
Files
• cp source destination – Copy file from source to
destination.
• mv source destination – Move or rename a file.
Deleting Files:
• rm filename – Remove a file.
• rmdir directoryname – Remove an empty directory.
• rm -r directoryname – Remove a directory and its
contents.
July 3, 2025 30
File Permissions and Ownership
• ls -l filename – View file permissions.
• chmod permissions filename – Change file permissions.
• chown owner filename – Change file owner.
• chgrp group filename – Change file group.
File System Navigation
• pwd – Print the current working directory.
• cd directoryname – Change directory.
• ls – List files and directories.
• ls -a – List all files, including hidden files.
July 3, 2025 31
Manipulating Files using System Calls
File Descriptor
UNIX system calls use file descriptors (integers
assigned to open files):
0 – Standard input (stdin)
1 – Standard output (stdout)
2 – Standard error (stderr)
July 3, 2025 32
System Calls for File Handling
• open(filename, mode) – Open a file.
• read(fd, buffer, count) – Read from a file.
• write(fd, buffer, count) – Write to a file.
• close(fd) – Close a file.
• lseek(fd, offset, whence) – Move file pointer.
File Creation and Deletion
creat(filename, mode) – Create a new file.
unlink(filename) – Delete a file.
July 3, 2025 33
Special File Operations
Symbolic and Hard Links
• ln source linkname – Create a hard link.
• ln -s source linkname – Create a symbolic link.
• ls -i filename – Display inode number.
File Compression
• gzip filename – Compress a file.
• gunzip filename.gz – Decompress a file.
• tar -cvf archive.tar files – Create a tar archive.
• tar -xvf archive.tar – Extract files from an archive.
July 3, 2025 34
Files in UNIX
• Understanding UNIX file structures and
file manipulation commands is crucial
for efficient system programming and
administration.
• Mastering these concepts allows users
to manage files effectively, automate
tasks, and interact with the system at a
low level.
July 3, 2025 35
Shell Scripting and Macro Processing
Introduction to Shell Scripting
July 3, 2025 37
Shell Variables and Operators
Types of Variables:
• Local Variables – Available only in the script they are defined in
• Global Variables – Available across multiple scripts and system-
wide
• Special Variables:
• $0 – Name of the script
• $1, $2, ... – Positional parameters
• $? – Exit status of last command
• $$ – Process ID of the script
• $* and $@ – List of arguments passed to the script
July 3, 2025 42
Variables and Data Types
• Variables store data values in shell scripts.
• #!/bin/bash
• NAME="John“
• echo "Hello, $NAME!"
User Input
Shell scripts can take user input using the read
command.
#!/bin/bash
read -p "Enter your name: " NAME
echo "Hello, $NAME!"
July 3, 2025 43
Conditional Statements
if-else Statement
• #!/bin/bash
• read -p "Enter a number: " num
• if [ $num -gt 10 ]; then
• echo "Number is greater than 10“
• else
• echo "Number is 10 or less“
• fi
July 3, 2025 44
Case Statement
• #!/bin/bash
• read -p "Enter a fruit: " fruit
• case $fruit in
• apple) echo "You chose apple." ;;
banana) echo "You chose
banana." ;; *) echo "Unknown fruit." ;;
• esac
July 3, 2025 45
Looping Constructs
For Loop
#!/bin/bash
for i in 1 2 3 4 5
do
echo "Iteration $i“
Done
July 3, 2025 46
While Loop
#!/bin/bash
count=1
while [ $count -le 5 ]
do
echo "Count: $count"
((count++))
done
July 3, 2025 47
Shell Programming Constructs
• Shell programming constructs are control structures
used in scripts.
Functions
• Functions are reusable blocks of code.
#!/bin/bash
function greet {
echo "Hello, $1!“
}
greet "Alice"
July 3, 2025 48
File Handling
• Reading and writing to files in shell scripts.
Process Management
• Checking running processes:
ps aux | grep bash
Killing a process:
kill -9 <PID>
July 3, 2025 49
Error Handling
#!/bin/bash
if ! ls /nonexistentfile 2>/dev/null; then
echo "Error: File not found!"
exit 1
fi
July 3, 2025 50
Macro Language and Macro-Processor
• What is a Macro?
• A macro is a predefined sequence of commands
or code snippets that can be invoked to perform a
specific task.
• Defining Macros in Shell Scripting
• Using aliases:
• alias ll='ls -la‘
Using functions:
function today {
date
} Note the use of pseudocode
July 3, 2025 51
Macro Processors
A macro processor transforms input text by expanding
macros before execution.
Common macro processors include:
• m4 – A general-purpose macro processor
• cpp – C preprocessor for macro expansion in C
programs
Example of Macro Processing with m4
Create a file test.m4:
define(NAME, "Alice")
Hello NAME!
Process it using:
m4 test.m4
July 3, 2025 52
Using Macros in Makefiles
CC=gcc
CFLAGS=-Wall
all:
$(CC) $(CFLAGS) program.c -o program
Process using:
m4 test.m4
Output:
25
July 3, 2025 54
Using Macros with Arguments in
Shell Scripting
• Functions with arguments:
#!/bin/bash
function add_numbers {
echo "Sum: $(($1 + $2))“
}
add_numbers 5 10
July 3, 2025 55
Macros in Assembly Language
Example in NASM:
%macro PRINT 2
mov edx, %2
mov ecx, %1
call printf
%endmacro
section .data
message db "Hello, World!",0
section .text
global main
extern printf
main:
PRINT message, 13
July 3, 2025 56
Bash Shell Programming codes with various
use cases
File Backup Script
• Use Case: Backup important files from a directory._ Application:
Automates daily document backups.
#!/bin/bash
# Directory to backup
SOURCE_DIR="$HOME/Documents"
BACKUP_DIR="$HOME/Backup"
DATE=$(date +%Y-%m-%d)
# Backup command
tar -czf "$BACKUP_DIR/backup-$DATE.tar.gz" "$SOURCE_DIR"
• #!/bin/bash
• THRESHOLD=80
• EMAIL="[email protected]"
July 3, 2025 58
User Account Creation Script
Use Case: Add multiple users from a text file.
Useful in academic or organizational onboarding systems:
#!/bin/bash
USER_FILE="userlist.txt"
July 3, 2025 59
Log File Archiver Script
Use Case: Automatically archive old log files.
System log management in enterprise environments.
• #!/bin/bash
• LOG_DIR="/var/log"
• ARCHIVE_DIR="/var/log/archive"
• mkdir -p "$ARCHIVE_DIR"
July 3, 2025 60
Interactive Menu Script
Use Case: Provide users with an interactive terminal menu.
User-friendly system interaction tools:
#!/bin/bash
while true; do
echo "Choose an option:"
echo "1. Show date"
echo "2. Show uptime"
echo "3. Show logged in users"
echo "4. Exit"
read -p "Enter your choice: " choice
case $choice in
1) date ;;
2) uptime ;;
3) who ;;
4) exit ;;
*) echo "Invalid option." ;;
esac
done
July 3, 2025 61
Check Service Status Script
Use Case: Check if a service (e.g., nginx) is running.
Useful for monitoring web and database services:
#!/bin/bash
SERVICE="nginx"
July 3, 2025 62
Simple Calculator ScriptUse Case: Command-line
calculator using expr
Demonstrates shell arithmetic operations:
#!/bin/bash
July 3, 2025 63
Simple System Update & Upgrade Script
• #!/bin/bash
July 3, 2025 64
Using Other macros with arguments in Shell
Scripting: reusable code blocks.
Macro-like behavior using functions and parameter expansion.
• Macro with Arguments (Using Functions):
#!/bin/bash
July 3, 2025 65
Macro to Calculate Sum of Two Numbers
• #!/bin/bash
• sum() {
• local result=$(( $1 + $2 ))
• echo "Sum: $result"
• }
• sum 10 25
July 3, 2025 66
Macro to Create a Backup File with Timestamp
• #!/bin/bash
• backup_file() {
• local file=$1
• local timestamp=$(date +%Y%m%d%H%M%S)
• cp "$file" "${file}_backup_$timestamp"
• echo "Backup created: ${file}_backup_$timestamp"
• }
• # Usage
• backup_file mydata.txt
July 3, 2025 67
Macro to Check Disk Space on Any Directory
• #!/bin/bash
• check_space() {
• echo "Checking disk space for directory: $1"
• du -sh "$1"
• }
• check_space /home
July 3, 2025 68
Macro to Greet Multiple People (Loop with Arguments)
• #!/bin/bash
• greet_all() {
• for name in "$@"
• do
• echo "Hello, $name!"
• done
• }
July 3, 2025 69
Macro_Summary Table
July 3, 2025 70
Conclusion
July 3, 2025 71
File Structures and Manipulation of
Files in UNIX
Algorithm 1: Add two numbers entered
by the user
July 3, 2025 72